[dpdk-dev] [PATCH] eal: make max interrupt vectors configurable

[dpdk-dev] [PATCH] eal: make max interrupt vectors configurable

[Pavan Nikhilesh Bhagavatula]

From: Pavan Nikhilesh <pbhagavatula@marvell.com>

Make max interrupt vectors configurable so that platforms can
choose interrupt vector limit.

Signed-off-by: Pavan Nikhilesh <pbhagavatula@marvell.com>
---
 config/common_base                                 | 1 +
 lib/librte_eal/common/include/rte_eal_interrupts.h | 2 +-
 2 files changed, 2 insertions(+), 1 deletion(-)

diff --git a/config/common_base b/config/common_base
index 0b09a9348..e84f8a233 100644
--- a/config/common_base
+++ b/config/common_base
@@ -68,6 +68,7 @@ CONFIG_RTE_MAX_LCORE=128
 CONFIG_RTE_MAX_NUMA_NODES=8
 CONFIG_RTE_MAX_HEAPS=32
 CONFIG_RTE_MAX_MEMSEG_LISTS=64
+CONFIG_RTE_MAX_INTERRUPT_VECTORS=32
 # each memseg list will be limited to either RTE_MAX_MEMSEG_PER_LIST pages
 # or RTE_MAX_MEM_MB_PER_LIST megabytes worth of memory, whichever is smaller
 CONFIG_RTE_MAX_MEMSEG_PER_LIST=8192
diff --git a/lib/librte_eal/common/include/rte_eal_interrupts.h b/lib/librte_eal/common/include/rte_eal_interrupts.h
index 9d302f412..19e44019f 100644
--- a/lib/librte_eal/common/include/rte_eal_interrupts.h
+++ b/lib/librte_eal/common/include/rte_eal_interrupts.h
@@ -17,7 +17,7 @@
 #ifndef _RTE_EAL_INTERRUPTS_H_
 #define _RTE_EAL_INTERRUPTS_H_
 
-#define RTE_MAX_RXTX_INTR_VEC_ID     32
+#define RTE_MAX_RXTX_INTR_VEC_ID      RTE_MAX_INTERRUPT_VECTORS
 #define RTE_INTR_VEC_ZERO_OFFSET      0
 #define RTE_INTR_VEC_RXTX_OFFSET      1
 
-- 
2.20.1

[dpdk-dev] [PATCH] eal: make max interrupt vectors configurable

[Pavan Nikhilesh Bhagavatula]

From: Pavan Nikhilesh <pbhagavatula@marvell.com>

Make max interrupt vectors configurable so that platforms can
choose interrupt vector limit.

Signed-off-by: Pavan Nikhilesh <pbhagavatula@marvell.com>
---
 config/common_base                                 | 1 +
 lib/librte_eal/common/include/rte_eal_interrupts.h | 2 +-
 2 files changed, 2 insertions(+), 1 deletion(-)

diff --git a/config/common_base b/config/common_base
index 0b09a9348..e84f8a233 100644
--- a/config/common_base
+++ b/config/common_base
@@ -68,6 +68,7 @@ CONFIG_RTE_MAX_LCORE=128
 CONFIG_RTE_MAX_NUMA_NODES=8
 CONFIG_RTE_MAX_HEAPS=32
 CONFIG_RTE_MAX_MEMSEG_LISTS=64
+CONFIG_RTE_MAX_INTERRUPT_VECTORS=32
 # each memseg list will be limited to either RTE_MAX_MEMSEG_PER_LIST pages
 # or RTE_MAX_MEM_MB_PER_LIST megabytes worth of memory, whichever is smaller
 CONFIG_RTE_MAX_MEMSEG_PER_LIST=8192
diff --git a/lib/librte_eal/common/include/rte_eal_interrupts.h b/lib/librte_eal/common/include/rte_eal_interrupts.h
index 9d302f412..19e44019f 100644
--- a/lib/librte_eal/common/include/rte_eal_interrupts.h
+++ b/lib/librte_eal/common/include/rte_eal_interrupts.h
@@ -17,7 +17,7 @@
 #ifndef _RTE_EAL_INTERRUPTS_H_
 #define _RTE_EAL_INTERRUPTS_H_
 
-#define RTE_MAX_RXTX_INTR_VEC_ID     32
+#define RTE_MAX_RXTX_INTR_VEC_ID      RTE_MAX_INTERRUPT_VECTORS
 #define RTE_INTR_VEC_ZERO_OFFSET      0
 #define RTE_INTR_VEC_RXTX_OFFSET      1
 
-- 
2.20.1

Re: [dpdk-dev] [PATCH] eal: make max interrupt vectors configurable

[Jerin Jacob Kollanukkaran]

On Tue, 2019-03-26 at 12:54 +0000, Pavan Nikhilesh Bhagavatula wrote:
> From: Pavan Nikhilesh <pbhagavatula@marvell.com>
> 
> Make max interrupt vectors configurable so that platforms can
> choose interrupt vector limit.
> 
> Signed-off-by: Pavan Nikhilesh <pbhagavatula@marvell.com>
> ---
>  config/common_base                                 

Please update meson config too.

Re: [dpdk-dev] [PATCH] eal: make max interrupt vectors configurable

[Jerin Jacob Kollanukkaran]

On Tue, 2019-03-26 at 12:54 +0000, Pavan Nikhilesh Bhagavatula wrote:
> From: Pavan Nikhilesh <pbhagavatula@marvell.com>
> 
> Make max interrupt vectors configurable so that platforms can
> choose interrupt vector limit.
> 
> Signed-off-by: Pavan Nikhilesh <pbhagavatula@marvell.com>
> ---
>  config/common_base                                 

Please update meson config too.

Re: [dpdk-dev] [PATCH] eal: make max interrupt vectors configurable

[Pavan Nikhilesh Bhagavatula]

> -----Original Message-----
> From: Jerin Jacob Kollanukkaran
> Sent: Tuesday, March 26, 2019 6:38 PM
> To: thomas@monjalon.net; Pavan Nikhilesh Bhagavatula
> <pbhagavatula@marvell.com>
> Cc: dev@dpdk.org
> Subject: Re: [dpdk-dev] [PATCH] eal: make max interrupt vectors
> configurable
> 
> On Tue, 2019-03-26 at 12:54 +0000, Pavan Nikhilesh Bhagavatula wrote:
> > From: Pavan Nikhilesh <pbhagavatula@marvell.com>
> >
> > Make max interrupt vectors configurable so that platforms can choose
> > interrupt vector limit.
> >
> > Signed-off-by: Pavan Nikhilesh <pbhagavatula@marvell.com>
> > ---
> >  config/common_base
> 
> Please update meson config too.

Will update meson in the next version.

Thanks,
Pavan.

Re: [dpdk-dev] [PATCH] eal: make max interrupt vectors configurable

[Pavan Nikhilesh Bhagavatula]

> -----Original Message-----
> From: Jerin Jacob Kollanukkaran
> Sent: Tuesday, March 26, 2019 6:38 PM
> To: thomas@monjalon.net; Pavan Nikhilesh Bhagavatula
> <pbhagavatula@marvell.com>
> Cc: dev@dpdk.org
> Subject: Re: [dpdk-dev] [PATCH] eal: make max interrupt vectors
> configurable
> 
> On Tue, 2019-03-26 at 12:54 +0000, Pavan Nikhilesh Bhagavatula wrote:
> > From: Pavan Nikhilesh <pbhagavatula@marvell.com>
> >
> > Make max interrupt vectors configurable so that platforms can choose
> > interrupt vector limit.
> >
> > Signed-off-by: Pavan Nikhilesh <pbhagavatula@marvell.com>
> > ---
> >  config/common_base
> 
> Please update meson config too.

Will update meson in the next version.

Thanks,
Pavan.

[dpdk-dev] [PATCH v2] eal: make max interrupt vectors configurable

[Pavan Nikhilesh Bhagavatula]

From: Pavan Nikhilesh <pbhagavatula@marvell.com>

Make max interrupt vectors configurable so that platforms can
choose interrupt vector limit.

Signed-off-by: Pavan Nikhilesh <pbhagavatula@marvell.com>
---
 v2 Changes:
 - Add defaults for meson build. (Jerin Jacob)

 config/common_base                                 | 1 +
 config/meson.build                                 | 1 +
 lib/librte_eal/common/include/rte_eal_interrupts.h | 2 +-
 3 files changed, 3 insertions(+), 1 deletion(-)

diff --git a/config/common_base b/config/common_base
index 0b09a9348..e84f8a233 100644
--- a/config/common_base
+++ b/config/common_base
@@ -68,6 +68,7 @@ CONFIG_RTE_MAX_LCORE=128
 CONFIG_RTE_MAX_NUMA_NODES=8
 CONFIG_RTE_MAX_HEAPS=32
 CONFIG_RTE_MAX_MEMSEG_LISTS=64
+CONFIG_RTE_MAX_INTERRUPT_VECTORS=32
 # each memseg list will be limited to either RTE_MAX_MEMSEG_PER_LIST pages
 # or RTE_MAX_MEM_MB_PER_LIST megabytes worth of memory, whichever is smaller
 CONFIG_RTE_MAX_MEMSEG_PER_LIST=8192
diff --git a/config/meson.build b/config/meson.build
index 0419607d3..68bb74e20 100644
--- a/config/meson.build
+++ b/config/meson.build
@@ -112,6 +112,7 @@ dpdk_conf.set('RTE_EAL_ALLOW_INV_SOCKET_ID', get_option('allow_invalid_socket_id
 dpdk_conf.set('RTE_MAX_VFIO_GROUPS', 64)
 dpdk_conf.set('RTE_DRIVER_MEMPOOL_BUCKET_SIZE_KB', 64)
 dpdk_conf.set('RTE_LIBRTE_DPAA2_USE_PHYS_IOVA', true)
+dpdk_conf.set('RTE_MAX_INTERRUPT_VECTORS', 32)


 compile_time_cpuflags = []
diff --git a/lib/librte_eal/common/include/rte_eal_interrupts.h b/lib/librte_eal/common/include/rte_eal_interrupts.h
index 9d302f412..19e44019f 100644
--- a/lib/librte_eal/common/include/rte_eal_interrupts.h
+++ b/lib/librte_eal/common/include/rte_eal_interrupts.h
@@ -17,7 +17,7 @@
 #ifndef _RTE_EAL_INTERRUPTS_H_
 #define _RTE_EAL_INTERRUPTS_H_

-#define RTE_MAX_RXTX_INTR_VEC_ID     32
+#define RTE_MAX_RXTX_INTR_VEC_ID      RTE_MAX_INTERRUPT_VECTORS
 #define RTE_INTR_VEC_ZERO_OFFSET      0
 #define RTE_INTR_VEC_RXTX_OFFSET      1

-
2.20.1

[dpdk-dev] [PATCH v2] eal: make max interrupt vectors configurable

[Pavan Nikhilesh Bhagavatula]

From: Pavan Nikhilesh <pbhagavatula@marvell.com>

Make max interrupt vectors configurable so that platforms can
choose interrupt vector limit.

Signed-off-by: Pavan Nikhilesh <pbhagavatula@marvell.com>
---
 v2 Changes:
 - Add defaults for meson build. (Jerin Jacob)

 config/common_base                                 | 1 +
 config/meson.build                                 | 1 +
 lib/librte_eal/common/include/rte_eal_interrupts.h | 2 +-
 3 files changed, 3 insertions(+), 1 deletion(-)

diff --git a/config/common_base b/config/common_base
index 0b09a9348..e84f8a233 100644
--- a/config/common_base
+++ b/config/common_base
@@ -68,6 +68,7 @@ CONFIG_RTE_MAX_LCORE=128
 CONFIG_RTE_MAX_NUMA_NODES=8
 CONFIG_RTE_MAX_HEAPS=32
 CONFIG_RTE_MAX_MEMSEG_LISTS=64
+CONFIG_RTE_MAX_INTERRUPT_VECTORS=32
 # each memseg list will be limited to either RTE_MAX_MEMSEG_PER_LIST pages
 # or RTE_MAX_MEM_MB_PER_LIST megabytes worth of memory, whichever is smaller
 CONFIG_RTE_MAX_MEMSEG_PER_LIST=8192
diff --git a/config/meson.build b/config/meson.build
index 0419607d3..68bb74e20 100644
--- a/config/meson.build
+++ b/config/meson.build
@@ -112,6 +112,7 @@ dpdk_conf.set('RTE_EAL_ALLOW_INV_SOCKET_ID', get_option('allow_invalid_socket_id
 dpdk_conf.set('RTE_MAX_VFIO_GROUPS', 64)
 dpdk_conf.set('RTE_DRIVER_MEMPOOL_BUCKET_SIZE_KB', 64)
 dpdk_conf.set('RTE_LIBRTE_DPAA2_USE_PHYS_IOVA', true)
+dpdk_conf.set('RTE_MAX_INTERRUPT_VECTORS', 32)


 compile_time_cpuflags = []
diff --git a/lib/librte_eal/common/include/rte_eal_interrupts.h b/lib/librte_eal/common/include/rte_eal_interrupts.h
index 9d302f412..19e44019f 100644
--- a/lib/librte_eal/common/include/rte_eal_interrupts.h
+++ b/lib/librte_eal/common/include/rte_eal_interrupts.h
@@ -17,7 +17,7 @@
 #ifndef _RTE_EAL_INTERRUPTS_H_
 #define _RTE_EAL_INTERRUPTS_H_

-#define RTE_MAX_RXTX_INTR_VEC_ID     32
+#define RTE_MAX_RXTX_INTR_VEC_ID      RTE_MAX_INTERRUPT_VECTORS
 #define RTE_INTR_VEC_ZERO_OFFSET      0
 #define RTE_INTR_VEC_RXTX_OFFSET      1

-
2.20.1

Re: [dpdk-dev] [PATCH v2] eal: make max interrupt vectors configurable

[Thomas Monjalon]

26/03/2019 14:21, Pavan Nikhilesh Bhagavatula:
> From: Pavan Nikhilesh <pbhagavatula@marvell.com>
> 
> Make max interrupt vectors configurable so that platforms can
> choose interrupt vector limit.

What is the impact of setting a big value?
Can we agree on a big enough value without introducing any config?

Re: [dpdk-dev] [PATCH v2] eal: make max interrupt vectors configurable

[Thomas Monjalon]

26/03/2019 14:21, Pavan Nikhilesh Bhagavatula:
> From: Pavan Nikhilesh <pbhagavatula@marvell.com>
> 
> Make max interrupt vectors configurable so that platforms can
> choose interrupt vector limit.

What is the impact of setting a big value?
Can we agree on a big enough value without introducing any config?

Re: [dpdk-dev] [PATCH v2] eal: make max interrupt vectors configurable

[Bruce Richardson]

On Tue, Mar 26, 2019 at 03:04:58PM +0100, Thomas Monjalon wrote:
> 26/03/2019 14:21, Pavan Nikhilesh Bhagavatula:
> > From: Pavan Nikhilesh <pbhagavatula@marvell.com>
> > 
> > Make max interrupt vectors configurable so that platforms can
> > choose interrupt vector limit.
> 
> What is the impact of setting a big value?
> Can we agree on a big enough value without introducing any config?
> 
Is it a value that needs to be set differently per-platform, perhaps?

Re: [dpdk-dev] [PATCH v2] eal: make max interrupt vectors configurable

[Bruce Richardson]

On Tue, Mar 26, 2019 at 03:04:58PM +0100, Thomas Monjalon wrote:
> 26/03/2019 14:21, Pavan Nikhilesh Bhagavatula:
> > From: Pavan Nikhilesh <pbhagavatula@marvell.com>
> > 
> > Make max interrupt vectors configurable so that platforms can
> > choose interrupt vector limit.
> 
> What is the impact of setting a big value?
> Can we agree on a big enough value without introducing any config?
> 
Is it a value that needs to be set differently per-platform, perhaps?

Re: [dpdk-dev] [EXT] Re: [PATCH v2] eal: make max interrupt vectors configurable

[Jerin Jacob Kollanukkaran]

On Tue, 2019-03-26 at 14:43 +0000, Bruce Richardson wrote:

+ Anatoly

> 
> -------------------------------------------------------------------
> ---
> On Tue, Mar 26, 2019 at 03:04:58PM +0100, Thomas Monjalon wrote:
> > 26/03/2019 14:21, Pavan Nikhilesh Bhagavatula:
> > > From: Pavan Nikhilesh <pbhagavatula@marvell.com>
> > > 
> > > Make max interrupt vectors configurable so that platforms can
> > > choose interrupt vector limit.
> > 
> > What is the impact of setting a big value?
> > Can we agree on a big enough value without introducing any config?
> > 
> Is it a value that needs to be set differently per-platform, perhaps?

MSI supports 32 interrupts, MSI-X can allocate up to 2048 interrupt as
per the PCIe Spec.

In terms of cost, a few KB of memory in rte_intr_handle in structure
and more file handlers for vfio.

I think, We can move to reasonable numbers as 256 or 512 considering
the latest PCIe multi function devices will have enough interrupts.

And I don't think, it is a platform value as common config needs to
work for distro build which is using the base config. Marvell has PCIe
device which has 256 MSIX vectors, Not sure about other PCIe card.

Re: [dpdk-dev] [EXT] Re: [PATCH v2] eal: make max interrupt vectors configurable

[Jerin Jacob Kollanukkaran]

On Tue, 2019-03-26 at 14:43 +0000, Bruce Richardson wrote:

+ Anatoly

> 
> -------------------------------------------------------------------
> ---
> On Tue, Mar 26, 2019 at 03:04:58PM +0100, Thomas Monjalon wrote:
> > 26/03/2019 14:21, Pavan Nikhilesh Bhagavatula:
> > > From: Pavan Nikhilesh <pbhagavatula@marvell.com>
> > > 
> > > Make max interrupt vectors configurable so that platforms can
> > > choose interrupt vector limit.
> > 
> > What is the impact of setting a big value?
> > Can we agree on a big enough value without introducing any config?
> > 
> Is it a value that needs to be set differently per-platform, perhaps?

MSI supports 32 interrupts, MSI-X can allocate up to 2048 interrupt as
per the PCIe Spec.

In terms of cost, a few KB of memory in rte_intr_handle in structure
and more file handlers for vfio.

I think, We can move to reasonable numbers as 256 or 512 considering
the latest PCIe multi function devices will have enough interrupts.

And I don't think, it is a platform value as common config needs to
work for distro build which is using the base config. Marvell has PCIe
device which has 256 MSIX vectors, Not sure about other PCIe card.

[dpdk-dev] [PATCH v3] eal: increase max number of interrupt vectors

[Pavan Nikhilesh Bhagavatula]

From: Pavan Nikhilesh <pbhagavatula@marvell.com>

MSI-X permits a device to allocate up to 2048 interrupts as per PCIe
spec.
Increase the max number of vectors to a reasonable value of 512.

Signed-off-by: Pavan Nikhilesh <pbhagavatula@marvell.com>
---
 v3 Changes:
 - Instead of making it a configurable option set max vectors to a reasonable
 value of 512.

 v2 Changes:
 - Add defaults for meson build. (Jerin Jacob)

 lib/librte_eal/common/include/rte_eal_interrupts.h | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/lib/librte_eal/common/include/rte_eal_interrupts.h b/lib/librte_eal/common/include/rte_eal_interrupts.h
index 9d302f412..b370c0d26 100644
--- a/lib/librte_eal/common/include/rte_eal_interrupts.h
+++ b/lib/librte_eal/common/include/rte_eal_interrupts.h
@@ -17,7 +17,7 @@
 #ifndef _RTE_EAL_INTERRUPTS_H_
 #define _RTE_EAL_INTERRUPTS_H_

-#define RTE_MAX_RXTX_INTR_VEC_ID     32
+#define RTE_MAX_RXTX_INTR_VEC_ID      512
 #define RTE_INTR_VEC_ZERO_OFFSET      0
 #define RTE_INTR_VEC_RXTX_OFFSET      1

--
2.21.0

[dpdk-dev] [PATCH v3] eal: increase max number of interrupt vectors

[Pavan Nikhilesh Bhagavatula]

From: Pavan Nikhilesh <pbhagavatula@marvell.com>

MSI-X permits a device to allocate up to 2048 interrupts as per PCIe
spec.
Increase the max number of vectors to a reasonable value of 512.

Signed-off-by: Pavan Nikhilesh <pbhagavatula@marvell.com>
---
 v3 Changes:
 - Instead of making it a configurable option set max vectors to a reasonable
 value of 512.

 v2 Changes:
 - Add defaults for meson build. (Jerin Jacob)

 lib/librte_eal/common/include/rte_eal_interrupts.h | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/lib/librte_eal/common/include/rte_eal_interrupts.h b/lib/librte_eal/common/include/rte_eal_interrupts.h
index 9d302f412..b370c0d26 100644
--- a/lib/librte_eal/common/include/rte_eal_interrupts.h
+++ b/lib/librte_eal/common/include/rte_eal_interrupts.h
@@ -17,7 +17,7 @@
 #ifndef _RTE_EAL_INTERRUPTS_H_
 #define _RTE_EAL_INTERRUPTS_H_

-#define RTE_MAX_RXTX_INTR_VEC_ID     32
+#define RTE_MAX_RXTX_INTR_VEC_ID      512
 #define RTE_INTR_VEC_ZERO_OFFSET      0
 #define RTE_INTR_VEC_RXTX_OFFSET      1

--
2.21.0

Re: [dpdk-dev] [PATCH v3] eal: increase max number of interrupt vectors

[Thomas Monjalon]

31/03/2019 16:55, Pavan Nikhilesh Bhagavatula:
> From: Pavan Nikhilesh <pbhagavatula@marvell.com>
> 
> MSI-X permits a device to allocate up to 2048 interrupts as per PCIe
> spec.
> Increase the max number of vectors to a reasonable value of 512.
> 
> Signed-off-by: Pavan Nikhilesh <pbhagavatula@marvell.com>

Applied, thanks

Re: [dpdk-dev] [PATCH v3] eal: increase max number of interrupt vectors

[Thomas Monjalon]

31/03/2019 16:55, Pavan Nikhilesh Bhagavatula:
> From: Pavan Nikhilesh <pbhagavatula@marvell.com>
> 
> MSI-X permits a device to allocate up to 2048 interrupts as per PCIe
> spec.
> Increase the max number of vectors to a reasonable value of 512.
> 
> Signed-off-by: Pavan Nikhilesh <pbhagavatula@marvell.com>

Applied, thanks