From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 20B8BA0C45; Wed, 20 Oct 2021 09:13:34 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 9DB6440150; Wed, 20 Oct 2021 09:13:33 +0200 (CEST) Received: from mail-ed1-f41.google.com (mail-ed1-f41.google.com [209.85.208.41]) by mails.dpdk.org (Postfix) with ESMTP id 1C5B640142 for ; Wed, 20 Oct 2021 09:13:32 +0200 (CEST) Received: by mail-ed1-f41.google.com with SMTP id w19so22475065edd.2 for ; Wed, 20 Oct 2021 00:13:32 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=KjgaYPfvV6no8vhVQL6iuXbpy1O1DooNOGZmU1TTCOE=; b=SmU3h0ZriSl4MqWOYxdqpYdM7ebB3VUzzX4KHH4IhPMjmVrKz78fJkjid3VDqO96WE Sdv5h16PXXsBrSRE896p6PmWFRFHpXgZT/sNvUCsIBk6Q2YfqvRh+JvVFi/Aay1ihDeS AJS2nCuTon75gupKKWmiakkt1b9A6YrAKqUibdiOvwe9t3E+42CuFAgWIXIAy8RcRx/j iEejJBr3hi7EUg9ky/gguLGQLdQhe9Tu4jvUlCwho94ZAQrZ3Lg4R+wYlB8ClwXnPx8Y /CPbVNlS5B8RsZFQfPOgTCsEfT8wUy5aCrDiFry9/JrF7nRBq+oSJ3mJBjbBRPVmfn87 A2mw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=KjgaYPfvV6no8vhVQL6iuXbpy1O1DooNOGZmU1TTCOE=; b=Y1BY2sHa8zraRvvyx7xMcEY5i5spLYQW4e8HFE7WUGMuIXAIs3YP2Uj+I59wX7KdL0 70vTBnuTMZcsfi2Y+n1pCc+CWmBTOUvjzwx6VqAfThbDODa7ys8qvVB8j9YyLrE9NCHS 18u0CYBrWFxAJ7vglOlsnAaI08xOvLUacHLDeQYSZJRKNdgW9KcrCDx8NdFnK7RIzNsg dfMRoNTBWOPkrb/k5JMeG/I1Rr1gMQLEwNarHbrCYEhQiVKycvszubHRsOScrOIYEZV3 Aikjas3REhaBFnhU1sD02gjFJbkMs21e5jADxOBjly9kl7dMcXuLmTr4LXChpUK7h2UH 4rxQ== X-Gm-Message-State: AOAM533NUShhddCyNS+G+ErVSDiDobYqx4hkv4ZiKPFtHICQFn4D997s swZWN6hGpaBgfAil2n+QAqOEiK64SaqXWihGTMaeGBI/MN3Opg== X-Google-Smtp-Source: ABdhPJxD4tsSpC8PzYpdxUhwW+zmApYLuNMvx2jqGpJUtlfThzitmWPbRPwZkalmVZxDTN9s2iahkSzbaQbdJhNqN04= X-Received: by 2002:a05:6402:5216:: with SMTP id s22mr62111635edd.167.1634714011742; Wed, 20 Oct 2021 00:13:31 -0700 (PDT) MIME-Version: 1.0 References: <1634306769-20123-1-git-send-email-tudor.cornea@gmail.com> In-Reply-To: From: Tudor Cornea Date: Wed, 20 Oct 2021 10:13:20 +0300 Message-ID: To: "Zhang, AlvinX" Cc: "Yigit, Ferruh" , "Zhang, Qi Z" , "Wang, Haiyue" , "dev@dpdk.org" Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.29 Subject: Re: [dpdk-dev] [PATCH] net/ixgbe: initialize max_rx_pkt_len if rlpml_set_vf fails X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Hi Ferruh, I have tested with the dpdk-next-net branch. It includes the commit 'ethdev: fix max Rx packet length' Setup: Hypervisor: VMware ESXi 6.0.0 PF ixgbe Driver: 3.7.13.7 (default PF driver installed with ESXi 6.0 and 6.5) NIC: Intel 82599 Guest OS : Ubuntu 20.04 It seems that with the recent changes testpmd still can't initialize the ports. EAL: Detected CPU lcores: 8 EAL: Detected NUMA nodes: 1 EAL: Detected static linkage of DPDK EAL: Multi-process socket /var/run/dpdk/rte/mp_socket EAL: Selected IOVA mode 'PA' EAL: No available 1048576 kB hugepages reported EAL: VFIO support initialized EAL: Probe PCI driver: net_ixgbe_vf (8086:10ed) device: 0000:0b:00.0 (socket 0) EAL: Probe PCI driver: net_ixgbe_vf (8086:10ed) device: 0000:13:00.0 (socket 0) TELEMETRY: No legacy callbacks, legacy socket not created Interactive-mode selected previous number of forwarding ports 2 - changed to number of configured ports 1 Error picking flow transfer proxy for port 0: Function not implemented - ignore Error picking flow transfer proxy for port 1: Function not implemented - ignore testpmd: create a new mbuf pool : n=171456, size=2176, socket=0 testpmd: preferred mempool ops selected: ring_mp_mc Warning! port-topology=paired and odd forward ports number, the last port will pair with itself. Configuring Port 0 (socket 0) ixgbevf_dev_rx_init(): Set max packet length to 1518 failed. ixgbevf_dev_start(): Unable to initialize RX hardware (-22) Fail to start port 0: Invalid argument Configuring Port 1 (socket 0) ixgbevf_dev_rx_init(): Set max packet length to 1518 failed. ixgbevf_dev_start(): Unable to initialize RX hardware (-22) Fail to start port 1: Invalid argument Please stop the ports first Done Error during enabling promiscuous mode for port 0: Operation not supported - ignore Error during enabling promiscuous mode for port 1: Operation not supported - ignore testpmd> start tx_first Not all ports were started I think failing to set set 'max_rx_pkt_len' after ixgbevf_rlpml_set_vf() call failed in ixgbevf_dev_set_mtu(), might have been one half of the problem. The other one is in ixgbevf_dev_rx_init(). The init function seems to return prematurely, and not initialize the ports. With the below patch (on top of net-next branch), I seem to be able to initialize the ports correctly, and send packets using testpmd. diff --git a/drivers/net/ixgbe/ixgbe_rxtx.c b/drivers/net/ixgbe/ixgbe_rxtx.c index b263dfe1d5..fdd057c789 100644 --- a/drivers/net/ixgbe/ixgbe_rxtx.c +++ b/drivers/net/ixgbe/ixgbe_rxtx.c @@ -5676,7 +5676,6 @@ ixgbevf_dev_rx_init(struct rte_eth_dev *dev) if (ixgbevf_rlpml_set_vf(hw, frame_size) != 0) { PMD_INIT_LOG(ERR, "Set max packet length to %d failed.", frame_size); - return -EINVAL; } /* Alvin, would it be acceptable to not return -EINVAL in this case while still printing an error, so that we can still debug mtu issues on 82599 ? If I recall correctly, the NIC has issues when the MTU of VFs is bigger than the MTU of the PFs. On my setup, however the MTUs have default values (1500). Should I rebase the patch on top of net-next ? Thanks, Tudor On Wed, 20 Oct 2021 at 06:08, Zhang, AlvinX wrote: > > -----Original Message----- > > From: Yigit, Ferruh > > Sent: Tuesday, October 19, 2021 8:58 PM > > To: Tudor Cornea ; Zhang, Qi Z > > > > Cc: Zhang, AlvinX ; Wang, Haiyue > > ; dev@dpdk.org > > Subject: Re: [dpdk-dev] [PATCH] net/ixgbe: initialize max_rx_pkt_len if > > rlpml_set_vf fails > > > > On 10/15/2021 3:20 PM, Tudor Cornea wrote: > > > Some of our customers use ESXi 6.0 or 6.5 servers, which could have > > > older versions of the PF ixgbe driver. > > > It seems that with a more recent version of the PMD driver, we are not > > > able to initialize 82599EB ports correctly. > > > This scenario seems to have worked with DPDK 19.11. > > > > > > Would it be possible to print a warning, while still allowing the > > > driver to initialize the ports ? > > There is a scenario that we initialize the port successful, with no any > error, but we cannot get any packets. > So keep printing error or warning and still allowing the driver to > initialize the port may be the best solution. > > > > > > > I was also thinking about the return code of ixgbevf_dev_set_mtu. > > > Do you think it would be more appropriate to return ENOTSUP or ENOSYS > > > instead of EINVAL ? > > If ixgbevf_dev_set_mtu fails, we have no way to know it because of invalid > value or not supported? > ixgbevf_dev_set_mtu returns one the these values: 0 or IXGBE_ERR_MBX > > > > > > > As a user, calling 'rte_eth_dev_mtu_set', I would expect an error like > > > EINVAL to suggest to me that the mtu value which I provided is > > > incorrect [1]. The 82599 NIC, however has some particularities related > > > to mtu, which could cause the operation to fail. I was thinking that > > > EINVAL might not be most descriptive error. > > > > > > [1] https://doc.dpdk.org/api/rte__ethdev_8h.html > > > > > > Thanks, > > > Tudor > > > > Hi Tudor, > > > > Can you please check if the patch is still after 'max_rx_pkt_len' > related changes > > in next-net? > > > > > > > > On Fri, 15 Oct 2021 at 17:06, Tudor Cornea > > wrote: > > > > > >> It seems that if the call to ixgbevf_rlpml_set_vf fails, we will not > > >> initialize dev_conf.rxmode.max_rx_pkt_len correctly anymore. > > >> > > >> This happens with a 82599EB NIC and a VMware ESXI 6.0 setup, and is > > >> causing VF the ports to fail to initialize > > >> > > >> We see the following error: > > >> ixgbevf_dev_rx_init(): Set max packet length to 1518 failed. > > >> > > >> Investigating over DPDK 19.11, it seems that the call still fails, > > >> but it doesn't exit prematurely, and max_rx_pkt_len is initialized in > > >> the respective case. > > >> > > >> On the ESXi server, we seem to have the following driver > > >> net-ixgbe: 3.7.13.7.14iov-20vmw.600.0.0.2494585 > > >> > > >> It seems that the behavior related to MTU setting has changed since > > >> the following commit: > > >> > > >> commit c77866a16904 ("net/ixgbe: detect failed VF MTU set") > > >> > > >> We would like to still be able to support older setups if possible, > > >> as we might have customers running ESXi 6.0 or 6.5, and these seem to > > >> have an older version of the PF driver as default. > > >> > > >> Signed-off-by: Tudor Cornea > > >> --- > > >> drivers/net/ixgbe/ixgbe_ethdev.c | 5 +++-- > > >> drivers/net/ixgbe/ixgbe_rxtx.c | 1 - > > >> 2 files changed, 3 insertions(+), 3 deletions(-) > > >> > > >> diff --git a/drivers/net/ixgbe/ixgbe_ethdev.c > > >> b/drivers/net/ixgbe/ixgbe_ethdev.c > > >> index 4dbe049..4301dfd 100644 > > >> --- a/drivers/net/ixgbe/ixgbe_ethdev.c > > >> +++ b/drivers/net/ixgbe/ixgbe_ethdev.c > > >> @@ -6369,6 +6369,9 @@ ixgbevf_dev_set_mtu(struct rte_eth_dev *dev, > > >> uint16_t mtu) > > >> return -EINVAL; > > >> } > > >> > > >> + /* update max frame size */ > > >> + dev->data->dev_conf.rxmode.max_rx_pkt_len = max_frame; > > >> + > > >> /* > > >> * When supported by the underlying PF driver, use the > > >> IXGBE_VF_SET_MTU > > >> * request of the version 2.0 of the mailbox API. > > >> @@ -6381,8 +6384,6 @@ ixgbevf_dev_set_mtu(struct rte_eth_dev *dev, > > >> uint16_t mtu) > > >> if (ixgbevf_rlpml_set_vf(hw, max_frame)) > > >> return -EINVAL; > > >> > > >> - /* update max frame size */ > > >> - dev->data->dev_conf.rxmode.max_rx_pkt_len = max_frame; > > >> return 0; > > >> } > > >> > > >> diff --git a/drivers/net/ixgbe/ixgbe_rxtx.c > > >> b/drivers/net/ixgbe/ixgbe_rxtx.c index 0ac89cb..02d9809 100644 > > >> --- a/drivers/net/ixgbe/ixgbe_rxtx.c > > >> +++ b/drivers/net/ixgbe/ixgbe_rxtx.c > > >> @@ -5677,7 +5677,6 @@ ixgbevf_dev_rx_init(struct rte_eth_dev *dev) > > >> (uint16_t)dev->data->dev_conf.rxmode.max_rx_pkt_len)) { > > >> PMD_INIT_LOG(ERR, "Set max packet length to %d > > failed.", > > >> > > dev->data->dev_conf.rxmode.max_rx_pkt_len); > > >> - return -EINVAL; > > >> } > > >> > > >> /* > > >> -- > > >> 2.7.4 > > >> > > >> > >