[PATCH 22.11] net/gve: fix RX buffer size alignment

[PATCH 22.11] net/gve: fix RX buffer size alignment

[Joshua Washington]

The GVE driver has RX buffer size alignment requirements which will
not always be respected when a user specifies an mbuf size. Assuming
that an mbuf size is greater than the DPDK recommended default
(2048 + 128), if the buffer size is not properly aligned with what the
device expects, the device will silently fail to create any transmit or
receive queues.

Because no queues are created, there is no network traffic for the DPDK
program, and errors like the following are returned when attempting to
destroy queues:

gve_adminq_parse_err(): AQ command failed with status -11
gve_stop_tx_queues(): failed to destroy txqs
gve_adminq_parse_err(): AQ command failed with status -11
gve_stop_rx_queues(): failed to destroy rxqs

This change aims to remedy this by restricting the RX receive buffer
sizes to valid sizes for the GQ queue format, including both alignment
and minimum and maximum supported buffer sizes.

Fixes: 4bec2d0b5572 ("net/gve: support queue operations")
Fixes: 1dc00f4fc74b ("net/gve: add Rx queue setup for DQO")
Cc: junfeng.guo@intel.com
Cc: stable@dpdk.org

Signed-off-by: Joshua Washington <joshwash@google.com>
Reviewed-by: Rushil Gupta <rushilg@google.com>
---
 drivers/net/gve/gve_ethdev.c | 2 +-
 drivers/net/gve/gve_ethdev.h | 4 +++-
 drivers/net/gve/gve_rx.c     | 7 ++++++-
 3 files changed, 10 insertions(+), 3 deletions(-)

diff --git a/drivers/net/gve/gve_ethdev.c b/drivers/net/gve/gve_ethdev.c
index 7872b4e476..0796d37760 100644
--- a/drivers/net/gve/gve_ethdev.c
+++ b/drivers/net/gve/gve_ethdev.c
@@ -275,7 +275,7 @@ gve_dev_info_get(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info)
 	dev_info->max_mac_addrs = 1;
 	dev_info->max_rx_queues = priv->max_nb_rxq;
 	dev_info->max_tx_queues = priv->max_nb_txq;
-	dev_info->min_rx_bufsize = GVE_MIN_BUF_SIZE;
+	dev_info->min_rx_bufsize = GVE_RX_MIN_BUF_SIZE_GQI;
 	dev_info->max_rx_pktlen = priv->max_mtu + RTE_ETHER_HDR_LEN;
 	dev_info->max_mtu = priv->max_mtu;
 	dev_info->min_mtu = RTE_ETHER_MIN_MTU;
diff --git a/drivers/net/gve/gve_ethdev.h b/drivers/net/gve/gve_ethdev.h
index 4123685b34..b7702a1249 100644
--- a/drivers/net/gve/gve_ethdev.h
+++ b/drivers/net/gve/gve_ethdev.h
@@ -28,7 +28,9 @@
 #define GVE_DEFAULT_TX_FREE_THRESH  256
 #define GVE_TX_MAX_FREE_SZ          512
 
-#define GVE_MIN_BUF_SIZE	    1024
+#define GVE_RX_BUF_ALIGN_GQI       2048
+#define GVE_RX_MIN_BUF_SIZE_GQI    2048
+#define GVE_RX_MAX_BUF_SIZE_GQI    4096
 
 /* A list of pages registered with the device during setup and used by a queue
  * as buffers
diff --git a/drivers/net/gve/gve_rx.c b/drivers/net/gve/gve_rx.c
index 518c9d109c..50f9f5c370 100644
--- a/drivers/net/gve/gve_rx.c
+++ b/drivers/net/gve/gve_rx.c
@@ -222,6 +222,7 @@ gve_rx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_id,
 	const struct rte_memzone *mz;
 	struct gve_rx_queue *rxq;
 	uint16_t free_thresh;
+	uint32_t mbuf_len;
 	int err = 0;
 
 	if (nb_desc != hw->rx_desc_cnt) {
@@ -265,7 +266,11 @@ gve_rx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_id,
 	rxq->hw = hw;
 	rxq->ntfy_addr = &hw->db_bar2[rte_be_to_cpu_32(hw->irq_dbs[rxq->ntfy_id].id)];
 
-	rxq->rx_buf_len = rte_pktmbuf_data_room_size(rxq->mpool) - RTE_PKTMBUF_HEADROOM;
+	mbuf_len =
+		rte_pktmbuf_data_room_size(rxq->mpool) - RTE_PKTMBUF_HEADROOM;
+	rxq->rx_buf_len =
+		RTE_MIN((uint16_t)GVE_RX_MAX_BUF_SIZE_GQI,
+			RTE_ALIGN_FLOOR(mbuf_len, GVE_RX_BUF_ALIGN_GQI));
 
 	/* Allocate software ring */
 	rxq->sw_ring = rte_zmalloc_socket("gve rx sw ring", sizeof(struct rte_mbuf *) * nb_desc,
-- 
2.43.0.472.g3155946c3a-goog

RE: [PATCH 22.11] net/gve: fix RX buffer size alignment

[Guo, Junfeng]

> -----Original Message-----
> From: Joshua Washington <joshwash@google.com>
> Sent: Wednesday, December 13, 2023 04:07
> To: Guo, Junfeng <junfeng.guo@intel.com>; Li, Xiaoyun
> <xiaoyun.li@intel.com>; Rushil Gupta <rushilg@google.com>; Joshua
> Washington <joshwash@google.com>; Jeroen de Borst
> <jeroendb@google.com>
> Cc: stable@dpdk.org; Xueming Li <xuemingl@nvidia.com>
> Subject: [PATCH 22.11] net/gve: fix RX buffer size alignment
> 
> The GVE driver has RX buffer size alignment requirements which will
> not always be respected when a user specifies an mbuf size. Assuming
> that an mbuf size is greater than the DPDK recommended default
> (2048 + 128), if the buffer size is not properly aligned with what the
> device expects, the device will silently fail to create any transmit or
> receive queues.
> 
> Because no queues are created, there is no network traffic for the DPDK
> program, and errors like the following are returned when attempting to
> destroy queues:
> 
> gve_adminq_parse_err(): AQ command failed with status -11
> gve_stop_tx_queues(): failed to destroy txqs
> gve_adminq_parse_err(): AQ command failed with status -11
> gve_stop_rx_queues(): failed to destroy rxqs
> 
> This change aims to remedy this by restricting the RX receive buffer
> sizes to valid sizes for the GQ queue format, including both alignment
> and minimum and maximum supported buffer sizes.
> 
> Fixes: 4bec2d0b5572 ("net/gve: support queue operations")
> Fixes: 1dc00f4fc74b ("net/gve: add Rx queue setup for DQO")
> Cc: junfeng.guo@intel.com
> Cc: stable@dpdk.org
> 
> Signed-off-by: Joshua Washington <joshwash@google.com>
> Reviewed-by: Rushil Gupta <rushilg@google.com>
> ---
>  drivers/net/gve/gve_ethdev.c | 2 +-
>  drivers/net/gve/gve_ethdev.h | 4 +++-
>  drivers/net/gve/gve_rx.c     | 7 ++++++-
>  3 files changed, 10 insertions(+), 3 deletions(-)
> 
> --
> 2.43.0.472.g3155946c3a-goog

Acked-by: Junfeng Guo <junfeng.guo@intel.com>

Regards,
Junfeng Guo

RE: [PATCH 22.11] net/gve: fix RX buffer size alignment

[Xueming(Steven) Li]

Hi all,

Thanks for the backporting, patch applied.

> -----Original Message-----
> From: Guo, Junfeng <junfeng.guo@intel.com>
> Sent: 12/13/2023 10:07
> To: Joshua Washington <joshwash@google.com>; Li, Xiaoyun
> <xiaoyun.li@intel.com>; Rushil Gupta <rushilg@google.com>; Jeroen de Borst
> <jeroendb@google.com>
> Cc: stable@dpdk.org; Xueming(Steven) Li <xuemingl@nvidia.com>
> Subject: RE: [PATCH 22.11] net/gve: fix RX buffer size alignment
> 
> 
> 
> > -----Original Message-----
> > From: Joshua Washington <joshwash@google.com>
> > Sent: Wednesday, December 13, 2023 04:07
> > To: Guo, Junfeng <junfeng.guo@intel.com>; Li, Xiaoyun
> > <xiaoyun.li@intel.com>; Rushil Gupta <rushilg@google.com>; Joshua
> > Washington <joshwash@google.com>; Jeroen de Borst
> > <jeroendb@google.com>
> > Cc: stable@dpdk.org; Xueming Li <xuemingl@nvidia.com>
> > Subject: [PATCH 22.11] net/gve: fix RX buffer size alignment
> >
> > The GVE driver has RX buffer size alignment requirements which will
> > not always be respected when a user specifies an mbuf size. Assuming
> > that an mbuf size is greater than the DPDK recommended default
> > (2048 + 128), if the buffer size is not properly aligned with what the
> > device expects, the device will silently fail to create any transmit
> > or receive queues.
> >
> > Because no queues are created, there is no network traffic for the
> > DPDK program, and errors like the following are returned when
> > attempting to destroy queues:
> >
> > gve_adminq_parse_err(): AQ command failed with status -11
> > gve_stop_tx_queues(): failed to destroy txqs
> > gve_adminq_parse_err(): AQ command failed with status -11
> > gve_stop_rx_queues(): failed to destroy rxqs
> >
> > This change aims to remedy this by restricting the RX receive buffer
> > sizes to valid sizes for the GQ queue format, including both alignment
> > and minimum and maximum supported buffer sizes.
> >
> > Fixes: 4bec2d0b5572 ("net/gve: support queue operations")
> > Fixes: 1dc00f4fc74b ("net/gve: add Rx queue setup for DQO")
> > Cc: junfeng.guo@intel.com
> > Cc: stable@dpdk.org
> >
> > Signed-off-by: Joshua Washington <joshwash@google.com>
> > Reviewed-by: Rushil Gupta <rushilg@google.com>
> > ---
> >  drivers/net/gve/gve_ethdev.c | 2 +-
> >  drivers/net/gve/gve_ethdev.h | 4 +++-
> >  drivers/net/gve/gve_rx.c     | 7 ++++++-
> >  3 files changed, 10 insertions(+), 3 deletions(-)
> >
> > --
> > 2.43.0.472.g3155946c3a-goog
> 
> Acked-by: Junfeng Guo <junfeng.guo@intel.com>
> 
> Regards,
> Junfeng Guo