From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id AF2AA43320 for ; Tue, 14 Nov 2023 00:12:15 +0100 (CET) Received: from mails.dpdk.org (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id A90BC402B3; Tue, 14 Nov 2023 00:12:15 +0100 (CET) Received: from mail-pl1-f201.google.com (mail-pl1-f201.google.com [209.85.214.201]) by mails.dpdk.org (Postfix) with ESMTP id D5CEF4027B for ; Tue, 14 Nov 2023 00:12:14 +0100 (CET) Received: by mail-pl1-f201.google.com with SMTP id d9443c01a7336-1cc23aa0096so55311085ad.2 for ; Mon, 13 Nov 2023 15:12:14 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1699917134; x=1700521934; darn=dpdk.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=LVomfWC+bu93sole4ZQE0Nja/v6i6FRVq0gmHXuimBM=; b=UaDiqPdkGOr801/LaZ26x2togLaPtVjVHRGjrXRUI1Gq+9V2hLBqyBTLKhlF/sTYzQ 2OZLxuuuTzikS0LBil7aVzMDvR53Ft2jDInOI1Ib/ca5UYvOSPsPXFcuvRrSP2LhOPeu jRHIceUPVVm3iVgeed8cvNsuqsBzp+fDycGf95YMeyiNVNl+gQAbb1JE+lhdEcJ0wm23 pCW7KHHDZyFOJYPcAzCiOQkIXzG7odk3mSfv+5fExHWG0DLxGltrs4cJzAhwIWPrKMcL IFMMaiogZoak6GmzFd1VwLzHtkf5XVlgmEF9Cnuz8w7OJulo5/tf8AzxKKZwgUWE6n2b ZGcg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1699917134; x=1700521934; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=LVomfWC+bu93sole4ZQE0Nja/v6i6FRVq0gmHXuimBM=; b=HAJlS3MAyvRC44gsGRkBdlsuylLyl1rg/NvM/XcHyoXRCSlr6f+JefPDtDGLbIhpb6 Mk83FK7jwOPsG9Iipcq9EKkAt7Ezxiaa7uc3e6Id7Xt/3jDiwnDYvVOZZZsLnA8456FN kuR9Ee8ITy/hss6JoUOAneJesTHxSsqYsX8XUTldCbwdAm7cTJXvnP0i7gEdJAF2RPwe 9ilCNrPmf72WsGJnEWxgn7a/yeXWesCpUiKBe/A5jk+lmsA4cEQazQAgYdiTarG8tg4v Wu5keZNeEHKTEAC4efdA2pzCDZnPYFl68wsfIlbRwbT3YHAEe9Id6RS/jv/6kKmEjhEH DTxw== X-Gm-Message-State: AOJu0YxUPrLRXtd5V23RUt+PnQVJnDi3GuMOfZWMjH3Q0ZvgD7XXMU7D Zs+1DikIHgiFV1f6iyzSMJpDqeC7y2KDEg== X-Google-Smtp-Source: AGHT+IFt4/lTJMGYyatvuIwZx9u6ez7QyC+sgES5PdOb6NTUfSPlUvgx2HT/5yF0t76aM7xOJ7f/EIXSPHNUBA== X-Received: from joshwash.sea.corp.google.com ([2620:15c:11c:202:35c2:a552:d23a:aba4]) (user=joshwash job=sendgmr) by 2002:a17:903:2581:b0:1cc:533e:f512 with SMTP id jb1-20020a170903258100b001cc533ef512mr176285plb.6.1699917133892; Mon, 13 Nov 2023 15:12:13 -0800 (PST) Date: Mon, 13 Nov 2023 15:12:06 -0800 In-Reply-To: <20231111003410.2950594-1-joshwash@google.com> Mime-Version: 1.0 References: <20231111003410.2950594-1-joshwash@google.com> X-Mailer: git-send-email 2.42.0.869.gea05f2083d-goog Message-ID: <20231113231206.376452-1-joshwash@google.com> Subject: [PATCH v2] net/gve: fix RX buffer size alignment From: Joshua Washington To: Junfeng Guo , Jeroen de Borst , Rushil Gupta , Joshua Washington , Xiaoyun Li Cc: dev@dpdk.org, stable@dpdk.org, Ferruh Yigit Content-Type: text/plain; charset="UTF-8" X-BeenThere: stable@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: patches for DPDK stable branches List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: stable-bounces@dpdk.org In GVE, both queue formats have RX buffer size alignment requirements which will not always be respected when a user specifies an mbuf size. Assuming that an mbuf size is greater than the DPDK recommended default (2048 + 128), if the buffer size is not properly aligned with what the device expects, the device will silently fail to create any transmit or receive queues. Because no queues are created, there is no network traffic for the DPDK program, and errors like the following are returned when attempting to destroy queues: gve_adminq_parse_err(): AQ command failed with status -11 gve_stop_tx_queues(): failed to destroy txqs gve_adminq_parse_err(): AQ command failed with status -11 gve_stop_rx_queues(): failed to destroy rxqs This change aims to remedy this by restricting the RX receive buffer sizes to valid sizes for their respective queue formats, including both alignment and minimum and maximum supported buffer sizes. Fixes: 4bec2d0b5572 ("net/gve: support queue operations") Fixes: 1dc00f4fc74b ("net/gve: add Rx queue setup for DQO") Cc: junfeng.guo@intel.com Cc: stable@dpdk.org Signed-off-by: Joshua Washington Reviewed-by: Rushil Gupta --- drivers/net/gve/gve_ethdev.c | 5 ++++- drivers/net/gve/gve_ethdev.h | 8 +++++++- drivers/net/gve/gve_rx.c | 7 ++++++- drivers/net/gve/gve_rx_dqo.c | 6 +++++- 4 files changed, 22 insertions(+), 4 deletions(-) diff --git a/drivers/net/gve/gve_ethdev.c b/drivers/net/gve/gve_ethdev.c index eb3bc7e151..43b4ab523d 100644 --- a/drivers/net/gve/gve_ethdev.c +++ b/drivers/net/gve/gve_ethdev.c @@ -296,7 +296,10 @@ gve_dev_info_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info) dev_info->max_mac_addrs = 1; dev_info->max_rx_queues = priv->max_nb_rxq; dev_info->max_tx_queues = priv->max_nb_txq; - dev_info->min_rx_bufsize = GVE_MIN_BUF_SIZE; + if (gve_is_gqi(priv)) + dev_info->min_rx_bufsize = GVE_RX_MIN_BUF_SIZE_GQI; + else + dev_info->min_rx_bufsize = GVE_RX_MIN_BUF_SIZE_DQO; dev_info->max_rx_pktlen = priv->max_mtu + RTE_ETHER_HDR_LEN; dev_info->max_mtu = priv->max_mtu; dev_info->min_mtu = RTE_ETHER_MIN_MTU; diff --git a/drivers/net/gve/gve_ethdev.h b/drivers/net/gve/gve_ethdev.h index 755ee8ad15..37f2b60845 100644 --- a/drivers/net/gve/gve_ethdev.h +++ b/drivers/net/gve/gve_ethdev.h @@ -20,7 +20,13 @@ #define GVE_DEFAULT_TX_RS_THRESH 32 #define GVE_TX_MAX_FREE_SZ 512 -#define GVE_MIN_BUF_SIZE 1024 +#define GVE_RX_BUF_ALIGN_DQO 128 +#define GVE_RX_MIN_BUF_SIZE_DQO 1024 +#define GVE_RX_MAX_BUF_SIZE_DQO ((16 * 1024) - GVE_RX_BUF_ALIGN_DQO) + +#define GVE_RX_BUF_ALIGN_GQI 2048 +#define GVE_RX_MIN_BUF_SIZE_GQI 2048 +#define GVE_RX_MAX_BUF_SIZE_GQI 4096 #define GVE_TX_CKSUM_OFFLOAD_MASK ( \ RTE_MBUF_F_TX_L4_MASK | \ diff --git a/drivers/net/gve/gve_rx.c b/drivers/net/gve/gve_rx.c index b8c92ccda0..36a1b73c65 100644 --- a/drivers/net/gve/gve_rx.c +++ b/drivers/net/gve/gve_rx.c @@ -301,6 +301,7 @@ gve_rx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_id, const struct rte_memzone *mz; struct gve_rx_queue *rxq; uint16_t free_thresh; + uint32_t mbuf_len; int err = 0; if (nb_desc != hw->rx_desc_cnt) { @@ -344,7 +345,11 @@ gve_rx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_id, rxq->hw = hw; rxq->ntfy_addr = &hw->db_bar2[rte_be_to_cpu_32(hw->irq_dbs[rxq->ntfy_id].id)]; - rxq->rx_buf_len = rte_pktmbuf_data_room_size(rxq->mpool) - RTE_PKTMBUF_HEADROOM; + mbuf_len = + rte_pktmbuf_data_room_size(rxq->mpool) - RTE_PKTMBUF_HEADROOM; + rxq->rx_buf_len = + RTE_MIN((uint16_t)GVE_RX_MAX_BUF_SIZE_GQI, + RTE_ALIGN_FLOOR(mbuf_len, GVE_RX_BUF_ALIGN_GQI)); /* Allocate software ring */ rxq->sw_ring = rte_zmalloc_socket("gve rx sw ring", sizeof(struct rte_mbuf *) * nb_desc, diff --git a/drivers/net/gve/gve_rx_dqo.c b/drivers/net/gve/gve_rx_dqo.c index 7e7ddac48e..422784e7e0 100644 --- a/drivers/net/gve/gve_rx_dqo.c +++ b/drivers/net/gve/gve_rx_dqo.c @@ -220,6 +220,7 @@ gve_rx_queue_setup_dqo(struct rte_eth_dev *dev, uint16_t queue_id, const struct rte_memzone *mz; struct gve_rx_queue *rxq; uint16_t free_thresh; + uint32_t mbuf_len; int err = 0; if (nb_desc != hw->rx_desc_cnt) { @@ -264,8 +265,11 @@ gve_rx_queue_setup_dqo(struct rte_eth_dev *dev, uint16_t queue_id, rxq->hw = hw; rxq->ntfy_addr = &hw->db_bar2[rte_be_to_cpu_32(hw->irq_dbs[rxq->ntfy_id].id)]; - rxq->rx_buf_len = + mbuf_len = rte_pktmbuf_data_room_size(rxq->mpool) - RTE_PKTMBUF_HEADROOM; + rxq->rx_buf_len = + RTE_MIN((uint16_t)GVE_RX_MAX_BUF_SIZE_DQO, + RTE_ALIGN_FLOOR(mbuf_len, GVE_RX_BUF_ALIGN_DQO)); /* Allocate software ring */ rxq->sw_ring = rte_zmalloc_socket("gve rx sw ring", -- 2.42.0.869.gea05f2083d-goog