This patch enables VF RSS for Niantic, which allow each VF having at most 4 queues. The actual queue number per VF depends on the number of VF: VF number from 1~32: 4 queues per VF; VF number from 33~max vf num: 2 queues per VF; On host, to enable VF RSS functionality, mq mode should be set as ETH_MQ_RX_VMDQ_RSS or ETH_MQ_RX_RSS mode, and SRIOV mode should be activated. It also needs config VF RSS information like hash function, RSS key, RSS key length. Changchun Ouyang (6): ixgbe: Code cleanup ixgbe: Negotiate VF API version ixgbe: Get VF queue number ether: Check VMDq RSS mode ixgbe: Config VF RSS testpmd: Set Rx VMDq RSS mode app/test-pmd/testpmd.c | 9 ++++ lib/librte_ether/rte_ethdev.c | 21 ++++++-- lib/librte_pmd_ixgbe/ixgbe_ethdev.h | 1 + lib/librte_pmd_ixgbe/ixgbe_pf.c | 75 ++++++++++++++++++++++++++++- lib/librte_pmd_ixgbe/ixgbe_rxtx.c | 95 +++++++++++++++++++++++++++---------- 5 files changed, 171 insertions(+), 30 deletions(-) -- 1.8.4.2
On Mon, Dec 15, 2014 at 10:56:59AM +0800, Ouyang Changchun wrote: > This patch enables VF RSS for Niantic, which allow each VF having at most 4 queues. > The actual queue number per VF depends on the number of VF: > VF number from 1~32: 4 queues per VF; > VF number from 33~max vf num: 2 queues per VF; > > On host, to enable VF RSS functionality, mq mode should be set as ETH_MQ_RX_VMDQ_RSS > or ETH_MQ_RX_RSS mode, and SRIOV mode should be activated. > It also needs config VF RSS information like hash function, RSS key, RSS key length. > Hi Changchun, are there limitations to this support, as I understood that that RSS support for VFs was not fully available in Niantic-based hardware? /Bruce > Changchun Ouyang (6): > ixgbe: Code cleanup > ixgbe: Negotiate VF API version > ixgbe: Get VF queue number > ether: Check VMDq RSS mode > ixgbe: Config VF RSS > testpmd: Set Rx VMDq RSS mode > > app/test-pmd/testpmd.c | 9 ++++ > lib/librte_ether/rte_ethdev.c | 21 ++++++-- > lib/librte_pmd_ixgbe/ixgbe_ethdev.h | 1 + > lib/librte_pmd_ixgbe/ixgbe_pf.c | 75 ++++++++++++++++++++++++++++- > lib/librte_pmd_ixgbe/ixgbe_rxtx.c | 95 +++++++++++++++++++++++++++---------- > 5 files changed, 171 insertions(+), 30 deletions(-) > > -- > 1.8.4.2 >
Hi Bruce,
> -----Original Message-----
> From: Richardson, Bruce
> Sent: Monday, December 15, 2014 6:55 PM
> To: Ouyang, Changchun
> Cc: dev@dpdk.org
> Subject: Re: [dpdk-dev] [PATCH 0/6] Enable VF RSS for Niantic
>
> On Mon, Dec 15, 2014 at 10:56:59AM +0800, Ouyang Changchun wrote:
> > This patch enables VF RSS for Niantic, which allow each VF having at most 4
> queues.
> > The actual queue number per VF depends on the number of VF:
> > VF number from 1~32: 4 queues per VF;
> > VF number from 33~max vf num: 2 queues per VF;
> >
> > On host, to enable VF RSS functionality, mq mode should be set as
> > ETH_MQ_RX_VMDQ_RSS or ETH_MQ_RX_RSS mode, and SRIOV mode
> should be activated.
> > It also needs config VF RSS information like hash function, RSS key, RSS key
> length.
> >
>
> Hi Changchun,
>
> are there limitations to this support, as I understood that that RSS support for
> VFs was not fully available in Niantic-based hardware?
>
The limitation here is RETA table, RSS key, Hash function are shared by pf and vf,
This is not like FVL, in FVL each VF may has its own RETA.
DPDK can setup 4 queues per vf while Linux ixgbe driver setup only 2 queues per vf.
This is a little advantage in DPDK over Linux driver .
Thanks
Changchun
On 12/18/14 12:11, Vlad Zolotarov wrote: > From: Changchun Ouyang <changchun.ouyang@intel.com> > > This patch enables VF RSS for Niantic, which allow each VF having at most 4 queues. > The actual queue number per VF depends on the number of VF: > VF number from 1~32: 4 queues per VF; > VF number from 33~max vf num: 2 queues per VF; > > On host, to enable VF RSS functionality, mq mode should be set as ETH_MQ_RX_VMDQ_RSS > or ETH_MQ_RX_RSS mode, and SRIOV mode should be activated. > It also needs config VF RSS information like hash function, RSS key, RSS key length. This patch series is missing a few things: 1. Taking into the consideration the number of Rx queues requested by a user in the rte_eth_dev_configure(). 2. dev->dev_ops->reta_query used by a rte_eth_dev_rss_reta_query() is still not initialized for a VF. Thus there is no way to query the RSS table contents. 3. rte_eth_dev_info_get() returns reta_size == 0 when called for a VF function. thanks, vlad > > Changchun Ouyang (6): > ixgbe: Code cleanup > ixgbe: Negotiate VF API version > ixgbe: Get VF queue number > ether: Check VMDq RSS mode > ixgbe: Config VF RSS > testpmd: Set Rx VMDq RSS mode > > app/test-pmd/testpmd.c | 9 ++++ > lib/librte_ether/rte_ethdev.c | 21 ++++++-- > lib/librte_pmd_ixgbe/ixgbe_ethdev.h | 1 + > lib/librte_pmd_ixgbe/ixgbe_pf.c | 75 ++++++++++++++++++++++++++++- > lib/librte_pmd_ixgbe/ixgbe_rxtx.c | 95 +++++++++++++++++++++++++++---------- > 5 files changed, 171 insertions(+), 30 deletions(-) > > --
My response as below. From: Vlad Zolotarov [mailto:vladz@cloudius-systems.com] Sent: Thursday, December 18, 2014 9:06 PM To: dev@dpdk.org; Ouyang, Changchun Subject: Re: [PATCH 0/6] Enable VF RSS for Niantic On 12/18/14 12:11, Vlad Zolotarov wrote: From: Changchun Ouyang <changchun.ouyang@intel.com><mailto:changchun.ouyang@intel.com> This patch enables VF RSS for Niantic, which allow each VF having at most 4 queues. The actual queue number per VF depends on the number of VF: VF number from 1~32: 4 queues per VF; VF number from 33~max vf num: 2 queues per VF; On host, to enable VF RSS functionality, mq mode should be set as ETH_MQ_RX_VMDQ_RSS or ETH_MQ_RX_RSS mode, and SRIOV mode should be activated. It also needs config VF RSS information like hash function, RSS key, RSS key length. This patch series is missing a few things: 1. Taking into the consideration the number of Rx queues requested by a user in the rte_eth_dev_configure(). Changchun: yes, will have a v2 to consider it, as I say in my previous response. 1. dev->dev_ops->reta_query used by a rte_eth_dev_rss_reta_query() is still not initialized for a VF. Thus there is no way to query the RSS table Changchun: do you mean query rss table on guest? Niantic doesn't have separate reta for a specific vf, the reta is shared by pf and all vf. Do you think of returning all reta contents for each vf make sense? Or any other insight here? 1. rte_eth_dev_info_get() returns reta_size == 0 when called for a VF function. Changchun: same as above. Your insight here are welcome here. thanks Changchun
On Dec 19, 2014 3:35 AM, "Ouyang, Changchun" <changchun.ouyang@intel.com> wrote: > > My response as below. > > > > From: Vlad Zolotarov [mailto:vladz@cloudius-systems.com] > Sent: Thursday, December 18, 2014 9:06 PM > To: dev@dpdk.org; Ouyang, Changchun > Subject: Re: [PATCH 0/6] Enable VF RSS for Niantic > > > > > > On 12/18/14 12:11, Vlad Zolotarov wrote: >> >> From: Changchun Ouyang <changchun.ouyang@intel.com> >> >> >> >> This patch enables VF RSS for Niantic, which allow each VF having at most 4 queues. >> >> The actual queue number per VF depends on the number of VF: >> >> VF number from 1~32: 4 queues per VF; >> >> VF number from 33~max vf num: 2 queues per VF; >> >> >> >> On host, to enable VF RSS functionality, mq mode should be set as ETH_MQ_RX_VMDQ_RSS >> >> or ETH_MQ_RX_RSS mode, and SRIOV mode should be activated. >> >> It also needs config VF RSS information like hash function, RSS key, RSS key length. > > > This patch series is missing a few things: > > Taking into the consideration the number of Rx queues requested by a user in the rte_eth_dev_configure(). > > Changchun: yes, will have a v2 to consider it, as I say in my previous response. I must have joined the mailing list after your v2 series. I'll take a look in the web for it. > > dev->dev_ops->reta_query used by a rte_eth_dev_rss_reta_query() is still not initialized for a VF. Thus there is no way to query the RSS table > > Changchun: do you mean query rss table on guest? Of course. Niantic doesn’t have separate reta for a specific vf, the reta is shared by pf and all vf. AFAIK the reta is statically divided between all available functions. So, first of all we need a way to know that is the size of a single VF partition in order to know how many RSS queues may be configured. Then we need an ability to query/set the contents of the corresponding reta partition like we can for PF. > > Do you think of returning all reta contents for each vf make sense? Or any other insight here? > > rte_eth_dev_info_get() returns reta_size == 0 when called for a VF function. > > Changchun: same as above. > > Your insight here are welcome here. > > thanks > > Changchun > >
Hi, I have v3 patch for this to resolve your comments, We can discuss on v3 patch if there are further comments. Thanks Changchun From: Vladislav Zolotarov [mailto:vladz@cloudius-systems.com] Sent: Sunday, December 21, 2014 4:02 PM To: Ouyang, Changchun Cc: dev@dpdk.org Subject: RE: [PATCH 0/6] Enable VF RSS for Niantic On Dec 19, 2014 3:35 AM, "Ouyang, Changchun" <changchun.ouyang@intel.com<mailto:changchun.ouyang@intel.com>> wrote: > > My response as below. > > > > From: Vlad Zolotarov [mailto:vladz@cloudius-systems.com<mailto:vladz@cloudius-systems.com>] > Sent: Thursday, December 18, 2014 9:06 PM > To: dev@dpdk.org<mailto:dev@dpdk.org>; Ouyang, Changchun > Subject: Re: [PATCH 0/6] Enable VF RSS for Niantic > > > > > > On 12/18/14 12:11, Vlad Zolotarov wrote: >> >> From: Changchun Ouyang <changchun.ouyang@intel.com<mailto:changchun.ouyang@intel.com>> >> >> >> >> This patch enables VF RSS for Niantic, which allow each VF having at most 4 queues. >> >> The actual queue number per VF depends on the number of VF: >> >> VF number from 1~32: 4 queues per VF; >> >> VF number from 33~max vf num: 2 queues per VF; >> >> >> >> On host, to enable VF RSS functionality, mq mode should be set as ETH_MQ_RX_VMDQ_RSS >> >> or ETH_MQ_RX_RSS mode, and SRIOV mode should be activated. >> >> It also needs config VF RSS information like hash function, RSS key, RSS key length. > > > This patch series is missing a few things: > > Taking into the consideration the number of Rx queues requested by a user in the rte_eth_dev_configure(). > > Changchun: yes, will have a v2 to consider it, as I say in my previous response. I must have joined the mailing list after your v2 series. I'll take a look in the web for it. > > dev->dev_ops->reta_query used by a rte_eth_dev_rss_reta_query() is still not initialized for a VF. Thus there is no way to query the RSS table > > Changchun: do you mean query rss table on guest? Of course. Niantic doesn’t have separate reta for a specific vf, the reta is shared by pf and all vf. AFAIK the reta is statically divided between all available functions. So, first of all we need a way to know that is the size of a single VF partition in order to know how many RSS queues may be configured. Then we need an ability to query/set the contents of the corresponding reta partition like we can for PF. > > Do you think of returning all reta contents for each vf make sense? Or any other insight here? > > rte_eth_dev_info_get() returns reta_size == 0 when called for a VF function. > > Changchun: same as above. > > Your insight here are welcome here. > > thanks > > Changchun > >
On 12/24/14 08:37, Ouyang, Changchun wrote: > > Hi, > > I have v3 patch for this to resolve your comments, > > We can discuss on v3 patch if there are further comments. > I'm looking at your v3 patchset right now... ;) > Thanks > > Changchun > > *From:*Vladislav Zolotarov [mailto:vladz@cloudius-systems.com] > *Sent:* Sunday, December 21, 2014 4:02 PM > *To:* Ouyang, Changchun > *Cc:* dev@dpdk.org > *Subject:* RE: [PATCH 0/6] Enable VF RSS for Niantic > > > On Dec 19, 2014 3:35 AM, "Ouyang, Changchun" > <changchun.ouyang@intel.com <mailto:changchun.ouyang@intel.com>> wrote: > > > > My response as below. > > > > > > > > From: Vlad Zolotarov [mailto:vladz@cloudius-systems.com > <mailto:vladz@cloudius-systems.com>] > > Sent: Thursday, December 18, 2014 9:06 PM > > To: dev@dpdk.org <mailto:dev@dpdk.org>; Ouyang, Changchun > > Subject: Re: [PATCH 0/6] Enable VF RSS for Niantic > > > > > > > > > > > > On 12/18/14 12:11, Vlad Zolotarov wrote: > >> > >> From: Changchun Ouyang <changchun.ouyang@intel.com > <mailto:changchun.ouyang@intel.com>> > >> > >> > >> > >> This patch enables VF RSS for Niantic, which allow each VF having > at most 4 queues. > >> > >> The actual queue number per VF depends on the number of VF: > >> > >> VF number from 1~32: 4 queues per VF; > >> > >> VF number from 33~max vf num: 2 queues per VF; > >> > >> > >> > >> On host, to enable VF RSS functionality, mq mode should be set as > ETH_MQ_RX_VMDQ_RSS > >> > >> or ETH_MQ_RX_RSS mode, and SRIOV mode should be activated. > >> > >> It also needs config VF RSS information like hash function, RSS > key, RSS key length. > > > > > > This patch series is missing a few things: > > > > Taking into the consideration the number of Rx queues requested by a > user in the rte_eth_dev_configure(). > > > > Changchun: yes, will have a v2 to consider it, as I say in my > previous response. > > I must have joined the mailing list after your v2 series. I'll take a > look in the web for it. > > > > > dev->dev_ops->reta_query used by a rte_eth_dev_rss_reta_query() is > still not initialized for a VF. Thus there is no way to query the RSS > table > > > > Changchun: do you mean query rss table on guest? > > Of course. > > Niantic doesn’t have separate reta for a specific vf, the reta is > shared by pf and all vf. > > AFAIK the reta is statically divided between all available functions. > > So, first of all we need a way to know that is the size of a single VF > partition in order to know how many RSS queues may be configured. > > Then we need an ability to query/set the contents of the corresponding > reta partition like we can for PF. > > > > > Do you think of returning all reta contents for each vf make sense? > Or any other insight here? > > > > rte_eth_dev_info_get() returns reta_size == 0 when called for a VF > function. > > > > Changchun: same as above. > > > > Your insight here are welcome here. > > > > thanks > > > > Changchun > > > > >