On 11/11/2023 12:34 AM, Joshua Washington wrote:
> In GVE, both queue formats have RX buffer size alignment requirements
> which are not respected whenever the mbuf size is greater than the
> minimum required by DPDK (2048 + 128).
>

Hi Joshua,

We don't have a way to inform application about the alignment
requirement, so drivers enforces these as you are doing in this patch.

But I am not clear with what is "minimum required by DPDK", since
application can provide smaller mbufs.
Also not clear why this alignment cause problem only with mbuf size
bigger than 2048 + 128 bytes. Can you please clarify?

> This causes the driver to break
> silently in initialization, and no queues are created, leading to no
> network traffic.
>
> This change aims to remedy this by restricting the RX receive buffer
> sizes to valid sizes for their respective queue formats.
>
> Fixes: 4bec2d0b5572 ("net/gve: support queue operations")
> Fixes: 1dc00f4fc74b ("net/gve: add Rx queue setup for DQO")
> Cc: junfeng.guo@intel.com
> Cc: stable@dpdk.org
>
> Signed-off-by: Joshua Washington <joshwash@google.com>
> Reviewed-by: Rushil Gupta <rushilg@google.com>

<...>

> @@ -337,6 +343,20 @@ gve_clear_device_rings_ok(struct gve_priv *priv)
>                               &priv->state_flags);
>  }
> 
> +static inline int
> +gve_validate_rx_buffer_size(struct gve_priv *priv, uint16_t rx_buffer_size)
> +{
> +     uint16_t min_rx_buffer_size = gve_is_gqi(priv) ?
> +             GVE_RX_MIN_BUF_SIZE_GQI : GVE_RX_MIN_BUF_SIZE_DQO;
> +     if (rx_buffer_size < min_rx_buffer_size) {
> +             PMD_DRV_LOG(ERR, "mbuf size must be at least %hu bytes",
> +                         min_rx_buffer_size);
> +             return -EINVAL;
> +     }
> +
>

When 'dev_info->min_rx_bufsize' set correctly, above check should be
done in ethdev level, can you please check 'rte_eth_check_rx_mempool()'.


> +     return 0;
> +}
> +
>  int
>  gve_rx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_id, uint16_t nb_desc,
>                  unsigned int socket_id, const struct rte_eth_rxconf *conf,
> diff --git a/drivers/net/gve/gve_rx.c b/drivers/net/gve/gve_rx.c
> index b8c92ccda0..0049c6428d 100644
> --- a/drivers/net/gve/gve_rx.c
> +++ b/drivers/net/gve/gve_rx.c
> @@ -301,6 +301,7 @@ gve_rx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_id,
>       const struct rte_memzone *mz;
>       struct gve_rx_queue *rxq;
>       uint16_t free_thresh;
> +     uint32_t mbuf_len;
>       int err = 0;
> 
>       if (nb_desc != hw->rx_desc_cnt) {
> @@ -344,7 +345,14 @@ gve_rx_queue_setup(struct rte_eth_dev *dev, uint16_t queue_id,
>       rxq->hw = hw;
>       rxq->ntfy_addr = &hw->db_bar2[rte_be_to_cpu_32(hw->irq_dbs[rxq->ntfy_id].id)];
> 
> -     rxq->rx_buf_len = rte_pktmbuf_data_room_size(rxq->mpool) - RTE_PKTMBUF_HEADROOM;
> +     mbuf_len =
> +             rte_pktmbuf_data_room_size(rxq->mpool) - RTE_PKTMBUF_HEADROOM;
> +     err = gve_validate_rx_buffer_size(hw, mbuf_len);
> +     if (err)
> +             goto err_rxq;
> +     rxq->rx_buf_len =
> +             RTE_MIN((uint16_t)GVE_RX_MAX_BUF_SIZE_GQI,
> +                     RTE_ALIGN_FLOOR(mbuf_len, GVE_RX_BUF_ALIGN_GQI));
>

Just for your info, this release 'dev_info.max_rx_bufsize' and ethdev
layer note added [1] if user provides mbuf size bigger than this value.
Ethdev layer not is mainly for memmory optimization, but above check is
required for driver.

[1]
https://git.dpdk.org/dpdk/commit/?id=75c7849a9dcca356985fdb87f2d11cae135dfb1a