From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id E6655A0350; Fri, 28 Jan 2022 18:58:39 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id D76294281F; Fri, 28 Jan 2022 18:58:39 +0100 (CET) Received: from alln-iport-7.cisco.com (alln-iport-7.cisco.com [173.37.142.94]) by mails.dpdk.org (Postfix) with ESMTP id ED7C740041 for ; Fri, 28 Jan 2022 18:58:38 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=cisco.com; i=@cisco.com; l=7316; q=dns/txt; s=iport; t=1643392719; x=1644602319; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=A49q3xzrns1QFFS97WpL08AreBvFyD8qSWYPMlxVWVI=; b=b07xjTnrkTFEma92nI4s8fk9WS3AR/gyfeL/e44sBQAncRLQa5qTyv6V 0ehb+E5FCPP9/NPnQYnRDkro5h1BNY2Be01i+Wo6v7YO1p2AHplbUrtNB wkOetAG/HflhgKDqp0EDdBEfZ1xTyHKvrvzrkEZuzcgs7c5WeeaoP1OhC s=; X-IronPort-AV: E=Sophos;i="5.88,324,1635206400"; d="scan'208";a="806796115" Received: from alln-core-5.cisco.com ([173.36.13.138]) by alln-iport-7.cisco.com with ESMTP/TLS/DHE-RSA-SEED-SHA; 28 Jan 2022 17:58:38 +0000 Received: from cisco.com (savbu-usnic-a.cisco.com [10.193.184.48]) by alln-core-5.cisco.com (8.15.2/8.15.2) with ESMTP id 20SHwcqE029910; Fri, 28 Jan 2022 17:58:38 GMT Received: by cisco.com (Postfix, from userid 392789) id EA00620F2003; Fri, 28 Jan 2022 09:58:37 -0800 (PST) From: John Daley To: ferruh.yigit@intel.com, arybchenko@solarflare.com Cc: dev@dpdk.org, John Daley , Hyong Youb Kim Subject: [PATCH v4 3/3] net/enic: support max descriptors allowed by adapter Date: Fri, 28 Jan 2022 09:58:13 -0800 Message-Id: <20220128175813.20775-3-johndale@cisco.com> X-Mailer: git-send-email 2.33.1 In-Reply-To: <20220128175813.20775-1-johndale@cisco.com> References: <20220127191029.30793-1-johndale@cisco.com> <20220128175813.20775-1-johndale@cisco.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Outbound-SMTP-Client: 10.193.184.48, savbu-usnic-a.cisco.com X-Outbound-Node: alln-core-5.cisco.com X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Newer VIC adapters have the max number of supported RX and TX descriptors in their configuration. Use these values as the maximums. Signed-off-by: John Daley Reviewed-by: Hyong Youb Kim --- drivers/net/enic/base/cq_enet_desc.h | 6 ++++- drivers/net/enic/enic_res.c | 20 ++++++++++++---- drivers/net/enic/enic_res.h | 6 +++-- drivers/net/enic/enic_rxtx.c | 35 +++++++++++++++++++--------- 4 files changed, 49 insertions(+), 18 deletions(-) diff --git a/drivers/net/enic/base/cq_enet_desc.h b/drivers/net/enic/base/cq_enet_desc.h index a34a4f5400..02db85b9a0 100644 --- a/drivers/net/enic/base/cq_enet_desc.h +++ b/drivers/net/enic/base/cq_enet_desc.h @@ -67,7 +67,8 @@ struct cq_enet_rq_desc_64 { uint16_t vlan; uint16_t checksum_fcoe; uint8_t flags; - uint8_t unused[48]; + uint8_t fetch_idx_flags; + uint8_t unused[47]; uint8_t type_color; }; @@ -92,6 +93,9 @@ struct cq_enet_rq_desc_64 { #define CQ_ENET_RQ_DESC_BYTES_WRITTEN_BITS 14 #define CQ_ENET_RQ_DESC_BYTES_WRITTEN_MASK \ ((1 << CQ_ENET_RQ_DESC_BYTES_WRITTEN_BITS) - 1) +#define CQ_ENET_RQ_DESC_FETCH_IDX_BITS 2 +#define CQ_ENET_RQ_DESC_FETCH_IDX_MASK \ + ((1 << CQ_ENET_RQ_DESC_FETCH_IDX_BITS) - 1) #define CQ_ENET_RQ_DESC_FLAGS_TRUNCATED (0x1 << 14) #define CQ_ENET_RQ_DESC_FLAGS_VLAN_STRIPPED (0x1 << 15) diff --git a/drivers/net/enic/enic_res.c b/drivers/net/enic/enic_res.c index 9cfb857939..caf773bab2 100644 --- a/drivers/net/enic/enic_res.c +++ b/drivers/net/enic/enic_res.c @@ -26,6 +26,7 @@ int enic_get_vnic_config(struct enic *enic) struct vnic_enet_config *c = &enic->config; int err; uint64_t sizes; + uint32_t max_rq_descs, max_wq_descs; err = vnic_dev_get_mac_addr(enic->vdev, enic->mac_addr); if (err) { @@ -57,6 +58,8 @@ int enic_get_vnic_config(struct enic *enic) GET_CONFIG(loop_tag); GET_CONFIG(num_arfs); GET_CONFIG(max_pkt_size); + GET_CONFIG(max_rq_ring); + GET_CONFIG(max_wq_ring); /* max packet size is only defined in newer VIC firmware * and will be 0 for legacy firmware and VICs @@ -101,20 +104,29 @@ int enic_get_vnic_config(struct enic *enic) ((enic->filter_actions & FILTER_ACTION_COUNTER_FLAG) ? "count " : "")); - c->wq_desc_count = RTE_MIN((uint32_t)ENIC_MAX_WQ_DESCS, + /* The max size of RQ and WQ rings are specified in 1500 series VICs and + * beyond. If they are not specified by the VIC or if 64B CQ descriptors + * are not being used, the max number of descriptors is 4096. + */ + max_wq_descs = (enic->cq64_request && c->max_wq_ring) ? c->max_wq_ring : + ENIC_LEGACY_MAX_WQ_DESCS; + c->wq_desc_count = RTE_MIN(max_wq_descs, RTE_MAX((uint32_t)ENIC_MIN_WQ_DESCS, c->wq_desc_count)); c->wq_desc_count &= 0xffffffe0; /* must be aligned to groups of 32 */ - - c->rq_desc_count = RTE_MIN((uint32_t)ENIC_MAX_RQ_DESCS, + max_rq_descs = (enic->cq64_request && c->max_rq_ring) ? c->max_rq_ring + : ENIC_LEGACY_MAX_WQ_DESCS; + c->rq_desc_count = RTE_MIN(max_rq_descs, RTE_MAX((uint32_t)ENIC_MIN_RQ_DESCS, c->rq_desc_count)); c->rq_desc_count &= 0xffffffe0; /* must be aligned to groups of 32 */ + dev_debug(NULL, "Max supported VIC descriptors: WQ:%u, RQ:%u\n", + max_wq_descs, max_rq_descs); c->intr_timer_usec = RTE_MIN(c->intr_timer_usec, vnic_dev_get_intr_coal_timer_max(enic->vdev)); dev_info(enic_get_dev(enic), "vNIC MAC addr " RTE_ETHER_ADDR_PRT_FMT - "wq/rq %d/%d mtu %d, max mtu:%d\n", + " wq/rq %d/%d mtu %d, max mtu:%d\n", enic->mac_addr[0], enic->mac_addr[1], enic->mac_addr[2], enic->mac_addr[3], enic->mac_addr[4], enic->mac_addr[5], c->wq_desc_count, c->rq_desc_count, diff --git a/drivers/net/enic/enic_res.h b/drivers/net/enic/enic_res.h index 34f15d5a42..ae979d52be 100644 --- a/drivers/net/enic/enic_res.h +++ b/drivers/net/enic/enic_res.h @@ -12,9 +12,11 @@ #include "vnic_rq.h" #define ENIC_MIN_WQ_DESCS 64 -#define ENIC_MAX_WQ_DESCS 4096 #define ENIC_MIN_RQ_DESCS 64 -#define ENIC_MAX_RQ_DESCS 4096 + +/* 1400 series VICs and prior all have 4K max, after that it's in the config */ +#define ENIC_LEGACY_MAX_WQ_DESCS 4096 +#define ENIC_LEGACY_MAX_RQ_DESCS 4096 /* A descriptor ring has a multiple of 32 descriptors */ #define ENIC_ALIGN_DESCS 32 diff --git a/drivers/net/enic/enic_rxtx.c b/drivers/net/enic/enic_rxtx.c index 33e96b480e..74a90694c7 100644 --- a/drivers/net/enic/enic_rxtx.c +++ b/drivers/net/enic/enic_rxtx.c @@ -84,6 +84,7 @@ enic_recv_pkts_common(void *rx_queue, struct rte_mbuf **rx_pkts, uint8_t packet_error; uint16_t ciflags; uint8_t tc; + uint16_t rq_idx_msbs = 0; max_rx--; @@ -94,17 +95,24 @@ enic_recv_pkts_common(void *rx_queue, struct rte_mbuf **rx_pkts, /* Get the cq descriptor and extract rq info from it */ cqd = *cqd_ptr; + /* - * The first 16B of 64B descriptor is identical to the - * 16B descriptor, except type_color. Copy type_color - * from the 64B descriptor into the 16B descriptor's - * field, so the code below can assume the 16B - * descriptor format. + * The first 16B of a 64B descriptor is identical to a 16B + * descriptor except for the type_color and fetch index. Extract + * fetch index and copy the type_color from the 64B to where it + * would be in a 16B descriptor so sebwequent code can run + * without further conditionals. */ - if (use_64b_desc) + if (use_64b_desc) { + rq_idx_msbs = (((volatile struct cq_enet_rq_desc_64 *) + cqd_ptr)->fetch_idx_flags + & CQ_ENET_RQ_DESC_FETCH_IDX_MASK) + << CQ_DESC_COMP_NDX_BITS; cqd.type_color = tc; + } rq_num = cqd.q_number & CQ_DESC_Q_NUM_MASK; - rq_idx = cqd.completed_index & CQ_DESC_COMP_NDX_MASK; + rq_idx = rq_idx_msbs + + (cqd.completed_index & CQ_DESC_COMP_NDX_MASK); rq = &enic->rq[rq_num]; rqd_ptr = ((struct rq_enet_desc *)rq->ring.descs) + rq_idx; @@ -362,14 +370,19 @@ static inline void enic_free_wq_bufs(struct vnic_wq *wq, uint16_t completed_index) { struct rte_mbuf *buf; - struct rte_mbuf *m, *free[ENIC_MAX_WQ_DESCS]; + struct rte_mbuf *m, *free[ENIC_LEGACY_MAX_WQ_DESCS]; unsigned int nb_to_free, nb_free = 0, i; struct rte_mempool *pool; unsigned int tail_idx; unsigned int desc_count = wq->ring.desc_count; - nb_to_free = enic_ring_sub(desc_count, wq->tail_idx, completed_index) - + 1; + /* + * On 1500 Series VIC and beyond, greater than ENIC_LEGACY_MAX_WQ_DESCS + * may be attempted to be freed. Cap it at ENIC_LEGACY_MAX_WQ_DESCS. + */ + nb_to_free = RTE_MIN(enic_ring_sub(desc_count, wq->tail_idx, + completed_index) + 1, + (uint32_t)ENIC_LEGACY_MAX_WQ_DESCS); tail_idx = wq->tail_idx; pool = wq->bufs[tail_idx]->pool; for (i = 0; i < nb_to_free; i++) { @@ -381,7 +394,7 @@ static inline void enic_free_wq_bufs(struct vnic_wq *wq, } if (likely(m->pool == pool)) { - RTE_ASSERT(nb_free < ENIC_MAX_WQ_DESCS); + RTE_ASSERT(nb_free < ENIC_LEGACY_MAX_WQ_DESCS); free[nb_free++] = m; } else { rte_mempool_put_bulk(pool, (void *)free, nb_free); -- 2.33.1