[dpdk-dev] i40e queues per VF

[dpdk-dev] i40e queues per VF

[Thomas Monjalon]

Hi,

When reading the documentation, it is not easy to understand
the capability of i40evf for the number of queues.

First, please could you explain why we need a build-time config option?
In the doc, there is neither justification nor tuning guidelines:

http://dpdk.org/doc/guides/nics/i40e.html#config-file-options
"
CONFIG_RTE_LIBRTE_I40E_QUEUE_NUM_PER_PF (default 64)
Number of queues reserved for PF.
CONFIG_RTE_LIBRTE_I40E_QUEUE_NUM_PER_VF (default 4)
Number of queues reserved for each SR-IOV VF.
"

I feel these are hard limits and should be some constants in the code,
not some build configuration options.

The other doc to look at is:
http://dpdk.org/doc/guides/nics/intel_vf.html#intel-fortville-10-40-gigabit-ethernet-controller-vf-infrastructure
"
Each VF can have a maximum of 16 queue pairs.
"

Do we agree that a queue pair is 1 Rx queue / 1 Tx queue?
Note: the concept of queue pairs in Intel VF should be explained somewhere.

Below, a different limitation is given:
"
The available queue number(at most 4) per VF depends on the total number
of pool, which is determined by the max number of VF at PF initialization
stage and the number of queue specified in config
"

So what is the real maximum of queue pairs? 4 or 16?
The datasheet talks about 16 queues. Is it 8 pairs?

Is there something to configure the number of queues when creating VF
with the kernel driver?

Re: [dpdk-dev] i40e queues per VF

[Xu, Qian Q]

Helin/Jingjing
Could you give some answers here? Thx. 

> -----Original Message-----
> From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Thomas Monjalon
> Sent: Tuesday, February 14, 2017 6:07 PM
> To: Zhang, Helin <helin.zhang@intel.com>; Wu, Jingjing <jingjing.wu@intel.com>
> Cc: Mcnamara, John <john.mcnamara@intel.com>; dev@dpdk.org
> Subject: [dpdk-dev] i40e queues per VF
> 
> Hi,
> 
> When reading the documentation, it is not easy to understand the capability of
> i40evf for the number of queues.
> 
> First, please could you explain why we need a build-time config option?
> In the doc, there is neither justification nor tuning guidelines:
> 
> http://dpdk.org/doc/guides/nics/i40e.html#config-file-options
> "
> CONFIG_RTE_LIBRTE_I40E_QUEUE_NUM_PER_PF (default 64) Number of
> queues reserved for PF.
> CONFIG_RTE_LIBRTE_I40E_QUEUE_NUM_PER_VF (default 4) Number of
> queues reserved for each SR-IOV VF.
> "
> 
> I feel these are hard limits and should be some constants in the code, not some
> build configuration options.
> 
> The other doc to look at is:
> http://dpdk.org/doc/guides/nics/intel_vf.html#intel-fortville-10-40-gigabit-
> ethernet-controller-vf-infrastructure
> "
> Each VF can have a maximum of 16 queue pairs.
> "
> 
> Do we agree that a queue pair is 1 Rx queue / 1 Tx queue?
> Note: the concept of queue pairs in Intel VF should be explained somewhere.
> 
> Below, a different limitation is given:
> "
> The available queue number(at most 4) per VF depends on the total number of
> pool, which is determined by the max number of VF at PF initialization stage and
> the number of queue specified in config "
> 
> So what is the real maximum of queue pairs? 4 or 16?
> The datasheet talks about 16 queues. Is it 8 pairs?
> 
> Is there something to configure the number of queues when creating VF with the
> kernel driver?

Re: [dpdk-dev] i40e queues per VF

[Wu, Jingjing]

> -----Original Message-----
> From: Thomas Monjalon [mailto:thomas.monjalon@6wind.com]
> Sent: Tuesday, February 14, 2017 6:07 PM
> To: Zhang, Helin <helin.zhang@intel.com>; Wu, Jingjing <jingjing.wu@intel.com>
> Cc: Mcnamara, John <john.mcnamara@intel.com>; dev@dpdk.org
> Subject: i40e queues per VF
> 
> Hi,
> 
> When reading the documentation, it is not easy to understand the capability of
> i40evf for the number of queues.
> 
> First, please could you explain why we need a build-time config option?
> In the doc, there is neither justification nor tuning guidelines:
> 
> http://dpdk.org/doc/guides/nics/i40e.html#config-file-options
> "
> CONFIG_RTE_LIBRTE_I40E_QUEUE_NUM_PER_PF (default 64) Number of
> queues reserved for PF.
> CONFIG_RTE_LIBRTE_I40E_QUEUE_NUM_PER_VF (default 4) Number of
> queues reserved for each SR-IOV VF.
> "

This number is used as initialization time to allocate queue number
for PF/VF for HW's queue pool. Will add more description in i40e.rst.
> 
> I feel these are hard limits and should be some constants in the code, not some
> build configuration options.
> 
> The other doc to look at is:
> http://dpdk.org/doc/guides/nics/intel_vf.html#intel-fortville-10-40-gigabit-
> ethernet-controller-vf-infrastructure
> "
> Each VF can have a maximum of 16 queue pairs.
> "
> 
> Do we agree that a queue pair is 1 Rx queue / 1 Tx queue?
> Note: the concept of queue pairs in Intel VF should be explained somewhere.
> 
Yes.
> Below, a different limitation is given:
> "
> The available queue number(at most 4) per VF depends on the total number of
> pool, which is determined by the max number of VF at PF initialization stage and
> the number of queue specified in config "
>
I think there may be some inconsistent description in  doc intel_vf.rst due to
Multiple kinds of NICs. We should correct them.
Thanks for pointing that.

> So what is the real maximum of queue pairs? 4 or 16?
> The datasheet talks about 16 queues. Is it 8 pairs?

That's is 16 queue pairs. 16 RX queues and 16 Tx queues.
> 
> Is there something to configure the number of queues when creating VF with the
> kernel driver?

In kernel driver, it seems at most only 4 queues are supported. That's
Why we add  build-time config option to make more queues are possible.


Thanks
Jingjing

Re: [dpdk-dev] i40e queues per VF

[Thomas Monjalon]

2017-02-16 13:58, Wu, Jingjing:
> From: Thomas Monjalon
> > 
> > Hi,
> > 
> > When reading the documentation, it is not easy to understand the capability of
> > i40evf for the number of queues.
> > 
> > First, please could you explain why we need a build-time config option?
> > In the doc, there is neither justification nor tuning guidelines:
> > 
> > http://dpdk.org/doc/guides/nics/i40e.html#config-file-options
> > "
> > CONFIG_RTE_LIBRTE_I40E_QUEUE_NUM_PER_PF (default 64) Number of
> > queues reserved for PF.
> > CONFIG_RTE_LIBRTE_I40E_QUEUE_NUM_PER_VF (default 4) Number of
> > queues reserved for each SR-IOV VF.
> > "
> 
> This number is used as initialization time to allocate queue number
> for PF/VF for HW's queue pool. Will add more description in i40e.rst.

The description "Number of queues reserved for each SR-IOV VF" seems
partially wrong. Please explain it is a queue pair.

> > I feel these are hard limits and should be some constants in the code, not some
> > build configuration options.
> > 
> > The other doc to look at is:
> > http://dpdk.org/doc/guides/nics/intel_vf.html#intel-fortville-10-40-gigabit-
> > ethernet-controller-vf-infrastructure
> > "
> > Each VF can have a maximum of 16 queue pairs.
> > "
> > 
> > Do we agree that a queue pair is 1 Rx queue / 1 Tx queue?
> > Note: the concept of queue pairs in Intel VF should be explained somewhere.
> > 
> Yes.
> > Below, a different limitation is given:
> > "
> > The available queue number(at most 4) per VF depends on the total number of
> > pool, which is determined by the max number of VF at PF initialization stage and
> > the number of queue specified in config "
> >
> I think there may be some inconsistent description in  doc intel_vf.rst due to
> Multiple kinds of NICs. We should correct them.
> Thanks for pointing that.
> 
> > So what is the real maximum of queue pairs? 4 or 16?
> > The datasheet talks about 16 queues. Is it 8 pairs?
> 
> That's is 16 queue pairs. 16 RX queues and 16 Tx queues.
> > 
> > Is there something to configure the number of queues when creating VF with the
> > kernel driver?
> 
> In kernel driver, it seems at most only 4 queues are supported. That's
> Why we add  build-time config option to make more queues are possible.

If we can create 16 queue pairs, why restrict the default configuration to 4?
Why is it a build-time config option?

Re: [dpdk-dev] i40e queues per VF

[Thomas Monjalon]

ping - still waiting for answers to below questions, please

2017-02-16 15:55, Thomas Monjalon:
> 2017-02-16 13:58, Wu, Jingjing:
> > From: Thomas Monjalon
> > > 
> > > Hi,
> > > 
> > > When reading the documentation, it is not easy to understand the capability of
> > > i40evf for the number of queues.
> > > 
> > > First, please could you explain why we need a build-time config option?
> > > In the doc, there is neither justification nor tuning guidelines:
> > > 
> > > http://dpdk.org/doc/guides/nics/i40e.html#config-file-options
> > > "
> > > CONFIG_RTE_LIBRTE_I40E_QUEUE_NUM_PER_PF (default 64) Number of
> > > queues reserved for PF.
> > > CONFIG_RTE_LIBRTE_I40E_QUEUE_NUM_PER_VF (default 4) Number of
> > > queues reserved for each SR-IOV VF.
> > > "
> > 
> > This number is used as initialization time to allocate queue number
> > for PF/VF for HW's queue pool. Will add more description in i40e.rst.
> 
> The description "Number of queues reserved for each SR-IOV VF" seems
> partially wrong. Please explain it is a queue pair.
> 
> > > I feel these are hard limits and should be some constants in the code, not some
> > > build configuration options.
> > > 
> > > The other doc to look at is:
> > > http://dpdk.org/doc/guides/nics/intel_vf.html#intel-fortville-10-40-gigabit-
> > > ethernet-controller-vf-infrastructure
> > > "
> > > Each VF can have a maximum of 16 queue pairs.
> > > "
> > > 
> > > Do we agree that a queue pair is 1 Rx queue / 1 Tx queue?
> > > Note: the concept of queue pairs in Intel VF should be explained somewhere.
> > > 
> > Yes.
> > > Below, a different limitation is given:
> > > "
> > > The available queue number(at most 4) per VF depends on the total number of
> > > pool, which is determined by the max number of VF at PF initialization stage and
> > > the number of queue specified in config "
> > >
> > I think there may be some inconsistent description in  doc intel_vf.rst due to
> > Multiple kinds of NICs. We should correct them.
> > Thanks for pointing that.
> > 
> > > So what is the real maximum of queue pairs? 4 or 16?
> > > The datasheet talks about 16 queues. Is it 8 pairs?
> > 
> > That's is 16 queue pairs. 16 RX queues and 16 Tx queues.
> > > 
> > > Is there something to configure the number of queues when creating VF with the
> > > kernel driver?
> > 
> > In kernel driver, it seems at most only 4 queues are supported. That's
> > Why we add  build-time config option to make more queues are possible.
> 
> If we can create 16 queue pairs, why restrict the default configuration to 4?
> Why is it a build-time config option?

Re: [dpdk-dev] i40e queues per VF

[Thomas Monjalon]

ping again

2017-03-08 12:25, Thomas Monjalon:
> ping - still waiting for answers to below questions, please
> 
> 2017-02-16 15:55, Thomas Monjalon:
> > 2017-02-16 13:58, Wu, Jingjing:
> > > From: Thomas Monjalon
> > > > 
> > > > Hi,
> > > > 
> > > > When reading the documentation, it is not easy to understand the capability of
> > > > i40evf for the number of queues.
> > > > 
> > > > First, please could you explain why we need a build-time config option?
> > > > In the doc, there is neither justification nor tuning guidelines:
> > > > 
> > > > http://dpdk.org/doc/guides/nics/i40e.html#config-file-options
> > > > "
> > > > CONFIG_RTE_LIBRTE_I40E_QUEUE_NUM_PER_PF (default 64) Number of
> > > > queues reserved for PF.
> > > > CONFIG_RTE_LIBRTE_I40E_QUEUE_NUM_PER_VF (default 4) Number of
> > > > queues reserved for each SR-IOV VF.
> > > > "
> > > 
> > > This number is used as initialization time to allocate queue number
> > > for PF/VF for HW's queue pool. Will add more description in i40e.rst.
> > 
> > The description "Number of queues reserved for each SR-IOV VF" seems
> > partially wrong. Please explain it is a queue pair.
> > 
> > > > I feel these are hard limits and should be some constants in the code, not some
> > > > build configuration options.
> > > > 
> > > > The other doc to look at is:
> > > > http://dpdk.org/doc/guides/nics/intel_vf.html#intel-fortville-10-40-gigabit-
> > > > ethernet-controller-vf-infrastructure
> > > > "
> > > > Each VF can have a maximum of 16 queue pairs.
> > > > "
> > > > 
> > > > Do we agree that a queue pair is 1 Rx queue / 1 Tx queue?
> > > > Note: the concept of queue pairs in Intel VF should be explained somewhere.
> > > > 
> > > Yes.
> > > > Below, a different limitation is given:
> > > > "
> > > > The available queue number(at most 4) per VF depends on the total number of
> > > > pool, which is determined by the max number of VF at PF initialization stage and
> > > > the number of queue specified in config "
> > > >
> > > I think there may be some inconsistent description in  doc intel_vf.rst due to
> > > Multiple kinds of NICs. We should correct them.
> > > Thanks for pointing that.
> > > 
> > > > So what is the real maximum of queue pairs? 4 or 16?
> > > > The datasheet talks about 16 queues. Is it 8 pairs?
> > > 
> > > That's is 16 queue pairs. 16 RX queues and 16 Tx queues.
> > > > 
> > > > Is there something to configure the number of queues when creating VF with the
> > > > kernel driver?
> > > 
> > > In kernel driver, it seems at most only 4 queues are supported. That's
> > > Why we add  build-time config option to make more queues are possible.
> > 
> > If we can create 16 queue pairs, why restrict the default configuration to 4?
> > Why is it a build-time config option?

Re: [dpdk-dev] i40e queues per VF

[Wu, Jingjing]

> > This number is used as initialization time to allocate queue number
> > for PF/VF for HW's queue pool. Will add more description in i40e.rst.
> 
> The description "Number of queues reserved for each SR-IOV VF" seems
> partially wrong. Please explain it is a queue pair.
> 
OK. Sure.

[......]

> > >
> > > Is there something to configure the number of queues when creating
> > > VF with the kernel driver?
> >
> > In kernel driver, it seems at most only 4 queues are supported. That's
> > Why we add  build-time config option to make more queues are possible.
> 
> If we can create 16 queue pairs, why restrict the default configuration to 4?
> Why is it a build-time config option?
If we allocate 16 queue pairs by default, then the number of VFs will be limited
Because of the entire queue resources on NIC.

Anyway, I will update doc on this part. Sorry for the delay because of trip.

Thanks
Jingjing