[PATCH 19.11] net/iavf: fix Rx queue interrupt setting

[PATCH 19.11] net/iavf: fix Rx queue interrupt setting

[Ke Zhang]

[ upstream commit a08f9cb698c3d4687765860e328dca727f7a516b ]

For Rx-Queue Interrupt Setting, when vf rx interrupt
disable(INTENA=0), there are two ways to write back
descriptor to host memory:

1)Set WB_ON_ITR bit 0 to Interrupt Dynamic Control Register:
Completed descriptors are posted to host memory according to
the internal descriptor cache policy (in other words when a
full cache line is available for write-back).

A internal descriptor size is 16 bytes or 32 bytes, a cache
line size is 64 bytes or 128 bytes from datasheet :
PCIe Global Config 2 - GLPCI_CNF2 (0x000BE004; RO)
so the full cache line could contains 4 packets, it means
Network card will send 4 packets to host when a full cache line
is available.

2)Set WB_ON_ITR bit 1 to Interrupt Dynamic Control Register:
Completed descriptors also trigger the ITR. Following ITR
expiration, all leftover completed descriptors are posted to
host memory.

Network card will send packet to host even if only one
descriptor is completed.

Changing 1) to 2) to make sure VF send the packet to host even
if there is only one rx packet is ready in hardware.

Fixes: d6bde6b5eae9 ("net/avf: enable Rx interrupt")
Cc: stable@dpdk.org

Signed-off-by: Ke Zhang <ke1x.zhang@intel.com>
---
 drivers/net/iavf/iavf_ethdev.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/net/iavf/iavf_ethdev.c b/drivers/net/iavf/iavf_ethdev.c
index 3cb02bd1f..00a47bf13 100644
--- a/drivers/net/iavf/iavf_ethdev.c
+++ b/drivers/net/iavf/iavf_ethdev.c
@@ -1209,7 +1209,7 @@ iavf_dev_rx_queue_intr_disable(struct rte_eth_dev *dev, uint16_t queue_id)
 
 	IAVF_WRITE_REG(hw,
 		      IAVFINT_DYN_CTLN1(msix_intr - IAVF_RX_VEC_START),
-		      0);
+		      IAVF_VFINT_DYN_CTLN1_WB_ON_ITR_MASK);
 
 	IAVF_WRITE_FLUSH(hw);
 	return 0;
-- 
2.25.1

Re: [PATCH 19.11] net/iavf: fix Rx queue interrupt setting

[Christian Ehrhardt]

On Wed, Jul 20, 2022 at 7:28 AM Ke Zhang <ke1x.zhang@intel.com> wrote:
>
> [ upstream commit a08f9cb698c3d4687765860e328dca727f7a516b ]

Thank you,
your patch was too late in regard to my original deadline and I was
then unavailable for a while.
In the meantime more patches came in and I do not want to waste any of
them just because they were late.

Your patch is applied to the WIP branch now, but currently testing of
-rc1 is going on which I do not want to disrupt.

If we need an -rc2 anyway or generally have the time to do an -rc2
without too much disruption it will be in 19.11.13, otherwise it is
already queued for 19.11.14

> For Rx-Queue Interrupt Setting, when vf rx interrupt
> disable(INTENA=0), there are two ways to write back
> descriptor to host memory:
>
> 1)Set WB_ON_ITR bit 0 to Interrupt Dynamic Control Register:
> Completed descriptors are posted to host memory according to
> the internal descriptor cache policy (in other words when a
> full cache line is available for write-back).
>
> A internal descriptor size is 16 bytes or 32 bytes, a cache
> line size is 64 bytes or 128 bytes from datasheet :
> PCIe Global Config 2 - GLPCI_CNF2 (0x000BE004; RO)
> so the full cache line could contains 4 packets, it means
> Network card will send 4 packets to host when a full cache line
> is available.
>
> 2)Set WB_ON_ITR bit 1 to Interrupt Dynamic Control Register:
> Completed descriptors also trigger the ITR. Following ITR
> expiration, all leftover completed descriptors are posted to
> host memory.
>
> Network card will send packet to host even if only one
> descriptor is completed.
>
> Changing 1) to 2) to make sure VF send the packet to host even
> if there is only one rx packet is ready in hardware.
>
> Fixes: d6bde6b5eae9 ("net/avf: enable Rx interrupt")
> Cc: stable@dpdk.org
>
> Signed-off-by: Ke Zhang <ke1x.zhang@intel.com>
> ---
>  drivers/net/iavf/iavf_ethdev.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/drivers/net/iavf/iavf_ethdev.c b/drivers/net/iavf/iavf_ethdev.c
> index 3cb02bd1f..00a47bf13 100644
> --- a/drivers/net/iavf/iavf_ethdev.c
> +++ b/drivers/net/iavf/iavf_ethdev.c
> @@ -1209,7 +1209,7 @@ iavf_dev_rx_queue_intr_disable(struct rte_eth_dev *dev, uint16_t queue_id)
>
>         IAVF_WRITE_REG(hw,
>                       IAVFINT_DYN_CTLN1(msix_intr - IAVF_RX_VEC_START),
> -                     0);
> +                     IAVF_VFINT_DYN_CTLN1_WB_ON_ITR_MASK);
>
>         IAVF_WRITE_FLUSH(hw);
>         return 0;
> --
> 2.25.1
>


-- 
Christian Ehrhardt
Senior Staff Engineer, Ubuntu Server
Canonical Ltd

RE: [PATCH 19.11] net/iavf: fix Rx queue interrupt setting

[Ali Alnubani]

> -----Original Message-----
> From: Christian Ehrhardt <christian.ehrhardt@canonical.com>
> Sent: Wednesday, August 3, 2022 12:58 PM
> To: Ke Zhang <ke1x.zhang@intel.com>
> Cc: stable@dpdk.org
> Subject: Re: [PATCH 19.11] net/iavf: fix Rx queue interrupt setting
> 
> On Wed, Jul 20, 2022 at 7:28 AM Ke Zhang <ke1x.zhang@intel.com> wrote:
> >
> > [ upstream commit a08f9cb698c3d4687765860e328dca727f7a516b ]
> 
> Thank you,
> your patch was too late in regard to my original deadline and I was
> then unavailable for a while.
> In the meantime more patches came in and I do not want to waste any of
> them just because they were late.
> 
> Your patch is applied to the WIP branch now, but currently testing of
> -rc1 is going on which I do not want to disrupt.
> 

Hello,

I believe this is causing the following build failure in 19.11.13-rc2:

"""
In file included from drivers/net/iavf/iavf_ethdev.c:13:
drivers/net/iavf/iavf_ethdev.c: In function 'iavf_dev_rx_queue_intr_disable':
drivers/net/iavf/iavf_ethdev.c:1212:9: error: 'IAVF_VFINT_DYN_CTLN1_WB_ON_ITR_MASK' undeclared (first use in this function); did you mean 'IAVFINT_DYN_CTLN1_WB_ON_ITR_MASK'?
 1212 |         IAVF_VFINT_DYN_CTLN1_WB_ON_ITR_MASK);
      |         ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
lib/librte_eal/common/include/arch/x86/rte_byteorder.h:74:30: note: in definition of macro 'rte_cpu_to_le_32'
   74 | #define rte_cpu_to_le_32(x) (x)
      |                              ^
drivers/net/iavf/base/iavf_osdep.h:116:2: note: in expansion of macro 'IAVF_PCI_REG_WRITE'
  116 |  IAVF_PCI_REG_WRITE(IAVF_PCI_REG_ADDR((hw), (reg)), (value))
      |  ^~~~~~~~~~~~~~~~~~
drivers/net/iavf/iavf_ethdev.c:1210:2: note: in expansion of macro 'IAVF_WRITE_REG'
 1210 |  IAVF_WRITE_REG(hw,
      |  ^~~~~~~~~~~~~~
drivers/net/iavf/iavf_ethdev.c:1212:9: note: each undeclared identifier is reported only once for each function it appears in
 1212 |         IAVF_VFINT_DYN_CTLN1_WB_ON_ITR_MASK);
      |         ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
lib/librte_eal/common/include/arch/x86/rte_byteorder.h:74:30: note: in definition of macro 'rte_cpu_to_le_32'
   74 | #define rte_cpu_to_le_32(x) (x)
      |                              ^
drivers/net/iavf/base/iavf_osdep.h:116:2: note: in expansion of macro 'IAVF_PCI_REG_WRITE'
  116 |  IAVF_PCI_REG_WRITE(IAVF_PCI_REG_ADDR((hw), (reg)), (value))
      |  ^~~~~~~~~~~~~~~~~~
drivers/net/iavf/iavf_ethdev.c:1210:2: note: in expansion of macro 'IAVF_WRITE_REG'
 1210 |  IAVF_WRITE_REG(hw,
      |  ^~~~~~~~~~~~~~
"""

I can reproduce on Ubuntu 20.04 with gcc 9.4.0.

- Ali

Re: [PATCH 19.11] net/iavf: fix Rx queue interrupt setting

[Christian Ehrhardt]

On Wed, Aug 3, 2022 at 1:40 PM Ali Alnubani <alialnu@nvidia.com> wrote:
>
> > -----Original Message-----
> > From: Christian Ehrhardt <christian.ehrhardt@canonical.com>
> > Sent: Wednesday, August 3, 2022 12:58 PM
> > To: Ke Zhang <ke1x.zhang@intel.com>
> > Cc: stable@dpdk.org
> > Subject: Re: [PATCH 19.11] net/iavf: fix Rx queue interrupt setting
> >
> > On Wed, Jul 20, 2022 at 7:28 AM Ke Zhang <ke1x.zhang@intel.com> wrote:
> > >
> > > [ upstream commit a08f9cb698c3d4687765860e328dca727f7a516b ]
> >
> > Thank you,
> > your patch was too late in regard to my original deadline and I was
> > then unavailable for a while.
> > In the meantime more patches came in and I do not want to waste any of
> > them just because they were late.
> >
> > Your patch is applied to the WIP branch now, but currently testing of
> > -rc1 is going on which I do not want to disrupt.
> >
>
> Hello,
>
> I believe this is causing the following build failure in 19.11.13-rc2:

I agree, I was trusting manual backports too much and releasing while
tests were still ongoing in the background.
Expect an -rc3 later today (this time with test builds completed
before tagging).

> """
> In file included from drivers/net/iavf/iavf_ethdev.c:13:
> drivers/net/iavf/iavf_ethdev.c: In function 'iavf_dev_rx_queue_intr_disable':
> drivers/net/iavf/iavf_ethdev.c:1212:9: error: 'IAVF_VFINT_DYN_CTLN1_WB_ON_ITR_MASK' undeclared (first use in this function); did you mean 'IAVFINT_DYN_CTLN1_WB_ON_ITR_MASK'?
>  1212 |         IAVF_VFINT_DYN_CTLN1_WB_ON_ITR_MASK);
>       |         ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> lib/librte_eal/common/include/arch/x86/rte_byteorder.h:74:30: note: in definition of macro 'rte_cpu_to_le_32'
>    74 | #define rte_cpu_to_le_32(x) (x)
>       |                              ^
> drivers/net/iavf/base/iavf_osdep.h:116:2: note: in expansion of macro 'IAVF_PCI_REG_WRITE'
>   116 |  IAVF_PCI_REG_WRITE(IAVF_PCI_REG_ADDR((hw), (reg)), (value))
>       |  ^~~~~~~~~~~~~~~~~~
> drivers/net/iavf/iavf_ethdev.c:1210:2: note: in expansion of macro 'IAVF_WRITE_REG'
>  1210 |  IAVF_WRITE_REG(hw,
>       |  ^~~~~~~~~~~~~~
> drivers/net/iavf/iavf_ethdev.c:1212:9: note: each undeclared identifier is reported only once for each function it appears in
>  1212 |         IAVF_VFINT_DYN_CTLN1_WB_ON_ITR_MASK);
>       |         ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> lib/librte_eal/common/include/arch/x86/rte_byteorder.h:74:30: note: in definition of macro 'rte_cpu_to_le_32'
>    74 | #define rte_cpu_to_le_32(x) (x)
>       |                              ^
> drivers/net/iavf/base/iavf_osdep.h:116:2: note: in expansion of macro 'IAVF_PCI_REG_WRITE'
>   116 |  IAVF_PCI_REG_WRITE(IAVF_PCI_REG_ADDR((hw), (reg)), (value))
>       |  ^~~~~~~~~~~~~~~~~~
> drivers/net/iavf/iavf_ethdev.c:1210:2: note: in expansion of macro 'IAVF_WRITE_REG'
>  1210 |  IAVF_WRITE_REG(hw,
>       |  ^~~~~~~~~~~~~~
> """
>
> I can reproduce on Ubuntu 20.04 with gcc 9.4.0.
>
> - Ali



-- 
Christian Ehrhardt
Senior Staff Engineer, Ubuntu Server
Canonical Ltd