* [dpdk-dev] [PATCH v2] net/ixgbe: fix l3fwd start failed on VF [not found] <20171102161031.167892-1-yanglong.wu@intel.com> @ 2017-11-20 2:00 ` Yanglong Wu 2017-11-20 2:25 ` [dpdk-dev] [PATCH v3] " Yanglong Wu 0 siblings, 1 reply; 10+ messages in thread From: Yanglong Wu @ 2017-11-20 2:00 UTC (permalink / raw) To: dev; +Cc: Yanglong Wu VF can't run in multi queue mode, if nb_q_per_pool was set as 1. The value of nb_q_per_pool is passed through to max_rx_q and max_tx_q in VF. So if nb_q_per_pool is equal to 1, max_rx_q and max_tx_q can't be more than 1 and VF multi queue mode will fail. Fixes: 27b609cbd1c6 (ethdev: move the multi-queue mode check to specific drivers) Signed-off-by: Yanglong Wu <yanglong.wu@intel.com> --- drivers/net/ixgbe/ixgbe_ethdev.c | 2 -- 1 file changed, 2 deletions(-) diff --git a/drivers/net/ixgbe/ixgbe_ethdev.c b/drivers/net/ixgbe/ixgbe_ethdev.c index 14b9c5303..ae9c44421 100644 --- a/drivers/net/ixgbe/ixgbe_ethdev.c +++ b/drivers/net/ixgbe/ixgbe_ethdev.c @@ -2222,8 +2222,6 @@ ixgbe_check_mq_mode(struct rte_eth_dev *dev) case ETH_MQ_RX_NONE: /* if nothing mq mode configure, use default scheme */ dev->data->dev_conf.rxmode.mq_mode = ETH_MQ_RX_VMDQ_ONLY; - if (RTE_ETH_DEV_SRIOV(dev).nb_q_per_pool > 1) - RTE_ETH_DEV_SRIOV(dev).nb_q_per_pool = 1; break; default: /* ETH_MQ_RX_DCB, ETH_MQ_RX_DCB_RSS or ETH_MQ_TX_DCB*/ /* SRIOV only works in VMDq enable mode */ -- 2.11.0 ^ permalink raw reply [flat|nested] 10+ messages in thread
* [dpdk-dev] [PATCH v3] net/ixgbe: fix l3fwd start failed on VF 2017-11-20 2:00 ` [dpdk-dev] [PATCH v2] net/ixgbe: fix l3fwd start failed on VF Yanglong Wu @ 2017-11-20 2:25 ` Yanglong Wu 2017-11-20 2:37 ` Yang, Zhiyong 2018-01-09 6:32 ` [dpdk-dev] [PATCH v4] " Yanglong Wu 0 siblings, 2 replies; 10+ messages in thread From: Yanglong Wu @ 2017-11-20 2:25 UTC (permalink / raw) To: dev; +Cc: Yanglong Wu VF can't run in multi queue mode, if nb_q_per_pool was set as 1. Nb_q_per_pool is passed through to max_rx_q and max_tx_q in VF. So if nb_q_per_pool is equal to 1, max_rx_q and max_tx_q can't be more than 1 and VF multi queue mode will fail. Fixes: 27b609cbd1c6 (ethdev: move the multi-queue mode check) Signed-off-by: Yanglong Wu <yanglong.wu@intel.com> --- drivers/net/ixgbe/ixgbe_ethdev.c | 2 -- 1 file changed, 2 deletions(-) diff --git a/drivers/net/ixgbe/ixgbe_ethdev.c b/drivers/net/ixgbe/ixgbe_ethdev.c index 14b9c5303..ae9c44421 100644 --- a/drivers/net/ixgbe/ixgbe_ethdev.c +++ b/drivers/net/ixgbe/ixgbe_ethdev.c @@ -2222,8 +2222,6 @@ ixgbe_check_mq_mode(struct rte_eth_dev *dev) case ETH_MQ_RX_NONE: /* if nothing mq mode configure, use default scheme */ dev->data->dev_conf.rxmode.mq_mode = ETH_MQ_RX_VMDQ_ONLY; - if (RTE_ETH_DEV_SRIOV(dev).nb_q_per_pool > 1) - RTE_ETH_DEV_SRIOV(dev).nb_q_per_pool = 1; break; default: /* ETH_MQ_RX_DCB, ETH_MQ_RX_DCB_RSS or ETH_MQ_TX_DCB*/ /* SRIOV only works in VMDq enable mode */ -- 2.11.0 ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [dpdk-dev] [PATCH v3] net/ixgbe: fix l3fwd start failed on VF 2017-11-20 2:25 ` [dpdk-dev] [PATCH v3] " Yanglong Wu @ 2017-11-20 2:37 ` Yang, Zhiyong 2018-01-09 6:32 ` [dpdk-dev] [PATCH v4] " Yanglong Wu 1 sibling, 0 replies; 10+ messages in thread From: Yang, Zhiyong @ 2017-11-20 2:37 UTC (permalink / raw) To: Wu, Yanglong, dev; +Cc: Wu, Yanglong Hi Yanglong, You should write something after SoB, which makes reviewer easier to know What changes are since last version. Thanks Zhiyong > -----Original Message----- > From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Yanglong Wu > Sent: Monday, November 20, 2017 10:26 AM > To: dev@dpdk.org > Cc: Wu, Yanglong <yanglong.wu@intel.com> > Subject: [dpdk-dev] [PATCH v3] net/ixgbe: fix l3fwd start failed on VF > > VF can't run in multi queue mode, if nb_q_per_pool was set as 1. > Nb_q_per_pool is passed through to max_rx_q and max_tx_q in VF. > So if nb_q_per_pool is equal to 1, max_rx_q and max_tx_q can't be more than 1 > and VF multi queue mode will fail. > > Fixes: 27b609cbd1c6 (ethdev: move the multi-queue mode check) > > Signed-off-by: Yanglong Wu <yanglong.wu@intel.com> > --- Changes in V3: ^ permalink raw reply [flat|nested] 10+ messages in thread
* [dpdk-dev] [PATCH v4] net/ixgbe: fix l3fwd start failed on VF 2017-11-20 2:25 ` [dpdk-dev] [PATCH v3] " Yanglong Wu 2017-11-20 2:37 ` Yang, Zhiyong @ 2018-01-09 6:32 ` Yanglong Wu 2018-01-11 9:48 ` Dai, Wei 1 sibling, 1 reply; 10+ messages in thread From: Yanglong Wu @ 2018-01-09 6:32 UTC (permalink / raw) To: dev, zhiyong.yang; +Cc: Yanglong Wu VF can't run in multi queue mode, if nb_q_per_pool was set as 1. Nb_q_per_pool is passed through to max_rx_q and max_tx_q in VF. So if nb_q_per_pool is equal to 1, max_rx_q and max_tx_q can't be more than 1 and VF multi queue mode will fail. Fixes: 27b609cbd1c6 (ethdev: move the multi-queue mode check) Signed-off-by: Yanglong Wu <yanglong.wu@intel.com> --- v1~v3: coding style issue fix --- v4: Rework according to comments --- drivers/net/ixgbe/ixgbe_ethdev.c | 2 -- 1 file changed, 2 deletions(-) diff --git a/drivers/net/ixgbe/ixgbe_ethdev.c b/drivers/net/ixgbe/ixgbe_ethdev.c index baaeee5d9..6af4f8b2b 100644 --- a/drivers/net/ixgbe/ixgbe_ethdev.c +++ b/drivers/net/ixgbe/ixgbe_ethdev.c @@ -2240,8 +2240,6 @@ ixgbe_check_mq_mode(struct rte_eth_dev *dev) case ETH_MQ_RX_NONE: /* if nothing mq mode configure, use default scheme */ dev->data->dev_conf.rxmode.mq_mode = ETH_MQ_RX_VMDQ_ONLY; - if (RTE_ETH_DEV_SRIOV(dev).nb_q_per_pool > 1) - RTE_ETH_DEV_SRIOV(dev).nb_q_per_pool = 1; break; default: /* ETH_MQ_RX_DCB, ETH_MQ_RX_DCB_RSS or ETH_MQ_TX_DCB*/ /* SRIOV only works in VMDq enable mode */ -- 2.11.0 ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [dpdk-dev] [PATCH v4] net/ixgbe: fix l3fwd start failed on VF 2018-01-09 6:32 ` [dpdk-dev] [PATCH v4] " Yanglong Wu @ 2018-01-11 9:48 ` Dai, Wei 2018-01-11 10:29 ` Wu, Yanglong 0 siblings, 1 reply; 10+ messages in thread From: Dai, Wei @ 2018-01-11 9:48 UTC (permalink / raw) To: Wu, Yanglong, dev, Yang, Zhiyong; +Cc: Wu, Yanglong According to 82599 datasheet, VF working mode is controlled by its host PF. If VF is working in RSS or DCB mode, it can use more than 1 queues. That also means its host PF should enable RSS or DCB. The PF Rx mode can be ETH_MQ_RX_VMDQ_RSS or ETH_MQ_RX_VMDQ_DCB. So I think that DPDK app with PF cause this issue, not ixgbe PMD itself. By the way, you can reply your patch mail to provide more info about how You reproduce this issue to easy others' reviewing and testing your patch. Thanks > -----Original Message----- > From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Yanglong Wu > Sent: Tuesday, January 9, 2018 2:32 PM > To: dev@dpdk.org; Yang, Zhiyong <zhiyong.yang@intel.com> > Cc: Wu, Yanglong <yanglong.wu@intel.com> > Subject: [dpdk-dev] [PATCH v4] net/ixgbe: fix l3fwd start failed on VF > > VF can't run in multi queue mode, if nb_q_per_pool was set as 1. > Nb_q_per_pool is passed through to max_rx_q and max_tx_q in VF. > So if nb_q_per_pool is equal to 1, max_rx_q and max_tx_q can't be more > than 1 and VF multi queue mode will fail. > > Fixes: 27b609cbd1c6 (ethdev: move the multi-queue mode check) > > Signed-off-by: Yanglong Wu <yanglong.wu@intel.com> > --- > v1~v3: > coding style issue fix > --- > v4: > Rework according to comments > --- > drivers/net/ixgbe/ixgbe_ethdev.c | 2 -- > 1 file changed, 2 deletions(-) > > diff --git a/drivers/net/ixgbe/ixgbe_ethdev.c > b/drivers/net/ixgbe/ixgbe_ethdev.c > index baaeee5d9..6af4f8b2b 100644 > --- a/drivers/net/ixgbe/ixgbe_ethdev.c > +++ b/drivers/net/ixgbe/ixgbe_ethdev.c > @@ -2240,8 +2240,6 @@ ixgbe_check_mq_mode(struct rte_eth_dev *dev) > case ETH_MQ_RX_NONE: > /* if nothing mq mode configure, use default scheme */ > dev->data->dev_conf.rxmode.mq_mode = > ETH_MQ_RX_VMDQ_ONLY; > - if (RTE_ETH_DEV_SRIOV(dev).nb_q_per_pool > 1) > - RTE_ETH_DEV_SRIOV(dev).nb_q_per_pool = 1; > break; > default: /* ETH_MQ_RX_DCB, ETH_MQ_RX_DCB_RSS or > ETH_MQ_TX_DCB*/ > /* SRIOV only works in VMDq enable mode */ > -- > 2.11.0 ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [dpdk-dev] [PATCH v4] net/ixgbe: fix l3fwd start failed on VF 2018-01-11 9:48 ` Dai, Wei @ 2018-01-11 10:29 ` Wu, Yanglong 2018-01-11 15:29 ` Wu, Yanglong 0 siblings, 1 reply; 10+ messages in thread From: Wu, Yanglong @ 2018-01-11 10:29 UTC (permalink / raw) To: Dai, Wei, dev Yes, this problem is caused by PF passing a wrong number of queue to VF. So basically, I removed model check of "case ETH_MQ_RX_NONE". It seem right after I do that but I'am also not very sure the changing is really right. So thank to your comments, I will redebug this problem. -----Original Message----- From: Dai, Wei Sent: Thursday, January 11, 2018 5:48 PM To: Wu, Yanglong <yanglong.wu@intel.com>; dev@dpdk.org; Yang, Zhiyong <zhiyong.yang@intel.com> Cc: Wu, Yanglong <yanglong.wu@intel.com> Subject: RE: [dpdk-dev] [PATCH v4] net/ixgbe: fix l3fwd start failed on VF According to 82599 datasheet, VF working mode is controlled by its host PF. If VF is working in RSS or DCB mode, it can use more than 1 queues. That also means its host PF should enable RSS or DCB. The PF Rx mode can be ETH_MQ_RX_VMDQ_RSS or ETH_MQ_RX_VMDQ_DCB. So I think that DPDK app with PF cause this issue, not ixgbe PMD itself. By the way, you can reply your patch mail to provide more info about how You reproduce this issue to easy others' reviewing and testing your patch. Thanks > -----Original Message----- > From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Yanglong Wu > Sent: Tuesday, January 9, 2018 2:32 PM > To: dev@dpdk.org; Yang, Zhiyong <zhiyong.yang@intel.com> > Cc: Wu, Yanglong <yanglong.wu@intel.com> > Subject: [dpdk-dev] [PATCH v4] net/ixgbe: fix l3fwd start failed on VF > > VF can't run in multi queue mode, if nb_q_per_pool was set as 1. > Nb_q_per_pool is passed through to max_rx_q and max_tx_q in VF. > So if nb_q_per_pool is equal to 1, max_rx_q and max_tx_q can't be more > than 1 and VF multi queue mode will fail. > > Fixes: 27b609cbd1c6 (ethdev: move the multi-queue mode check) > > Signed-off-by: Yanglong Wu <yanglong.wu@intel.com> > --- > v1~v3: > coding style issue fix > --- > v4: > Rework according to comments > --- > drivers/net/ixgbe/ixgbe_ethdev.c | 2 -- > 1 file changed, 2 deletions(-) > > diff --git a/drivers/net/ixgbe/ixgbe_ethdev.c > b/drivers/net/ixgbe/ixgbe_ethdev.c > index baaeee5d9..6af4f8b2b 100644 > --- a/drivers/net/ixgbe/ixgbe_ethdev.c > +++ b/drivers/net/ixgbe/ixgbe_ethdev.c > @@ -2240,8 +2240,6 @@ ixgbe_check_mq_mode(struct rte_eth_dev *dev) > case ETH_MQ_RX_NONE: > /* if nothing mq mode configure, use default scheme */ > dev->data->dev_conf.rxmode.mq_mode = ETH_MQ_RX_VMDQ_ONLY; > - if (RTE_ETH_DEV_SRIOV(dev).nb_q_per_pool > 1) > - RTE_ETH_DEV_SRIOV(dev).nb_q_per_pool = 1; > break; > default: /* ETH_MQ_RX_DCB, ETH_MQ_RX_DCB_RSS or ETH_MQ_TX_DCB*/ > /* SRIOV only works in VMDq enable mode */ > -- > 2.11.0 ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [dpdk-dev] [PATCH v4] net/ixgbe: fix l3fwd start failed on VF 2018-01-11 10:29 ` Wu, Yanglong @ 2018-01-11 15:29 ` Wu, Yanglong 2018-01-12 2:38 ` Wu, Yanglong 0 siblings, 1 reply; 10+ messages in thread From: Wu, Yanglong @ 2018-01-11 15:29 UTC (permalink / raw) To: Wu, Yanglong, Dai, Wei, dev Hi daiwei: After I double checked with this bug, the problem can be described as follow: When we start PF using testpmd like this: testpmd -c 0x3 -n 4 --socket-mem=1024,1024 --file-prefix=pf -w 0000:05:00.0 -w 0000:05:00.1 -- -i. The number of queue signed to each port would be 1 and the rxmode.mq_mode of ixgbe eth_dev would be set as ETH_MQ_RX_NONE. And this config info is make sense, so until now there is no problem, right? And according to the code: case ETH_MQ_RX_NONE: > /* if nothing mq mode configure, use default scheme */ > dev->data->dev_conf.rxmode.mq_mode = ETH_MQ_RX_VMDQ_ONLY; > - if (RTE_ETH_DEV_SRIOV(dev).nb_q_per_pool > 1) > - RTE_ETH_DEV_SRIOV(dev).nb_q_per_pool = 1; > break; The nb_q_per_pool of (RTE_ETH_DEV_SRIOV(dev).nb_q_per_pool would be set as 1, that means for VF the number of max queue can be used would be only 1. And next we start VF using l3fwd: l3fwd/build/l3fwd -c 0xc0000 -n 4 -w 0000:05:10.0 -w 0000:05:10.1 -- -p 0x3 --config '(0,0,18),(1,0,19)' --parse-ptype The number of rx_q equal to 1 and there is no problem. But, in l3fwd, the number of tx_q would equal to the number of core used in l3fwd, so here number of tx_q equal to 2 but as described above, the max number of queue can be used in VF is just only 1. So there is problem and l3fwd will check failed. So there is two ways to fix this bug, the first is using this patch, don't assign RTE_ETH_DEV_SRIOV(dev).nb_q_per_pool as 1 when mq_mode = ETH_MQ_RX_VMDQ_ONLY. And the second is that the number of tx_q would not assigned as the number of cores used in l3fwd. Which one do you think is make sense? Yanglong -----Original Message----- From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Wu, Yanglong Sent: Thursday, January 11, 2018 6:30 PM To: Dai, Wei <wei.dai@intel.com>; dev@dpdk.org Subject: Re: [dpdk-dev] [PATCH v4] net/ixgbe: fix l3fwd start failed on VF Yes, this problem is caused by PF passing a wrong number of queue to VF. So basically, I removed model check of "case ETH_MQ_RX_NONE". It seem right after I do that but I'am also not very sure the changing is really right. So thank to your comments, I will redebug this problem. -----Original Message----- From: Dai, Wei Sent: Thursday, January 11, 2018 5:48 PM To: Wu, Yanglong <yanglong.wu@intel.com>; dev@dpdk.org; Yang, Zhiyong <zhiyong.yang@intel.com> Cc: Wu, Yanglong <yanglong.wu@intel.com> Subject: RE: [dpdk-dev] [PATCH v4] net/ixgbe: fix l3fwd start failed on VF According to 82599 datasheet, VF working mode is controlled by its host PF. If VF is working in RSS or DCB mode, it can use more than 1 queues. That also means its host PF should enable RSS or DCB. The PF Rx mode can be ETH_MQ_RX_VMDQ_RSS or ETH_MQ_RX_VMDQ_DCB. So I think that DPDK app with PF cause this issue, not ixgbe PMD itself. By the way, you can reply your patch mail to provide more info about how You reproduce this issue to easy others' reviewing and testing your patch. Thanks > -----Original Message----- > From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Yanglong Wu > Sent: Tuesday, January 9, 2018 2:32 PM > To: dev@dpdk.org; Yang, Zhiyong <zhiyong.yang@intel.com> > Cc: Wu, Yanglong <yanglong.wu@intel.com> > Subject: [dpdk-dev] [PATCH v4] net/ixgbe: fix l3fwd start failed on VF > > VF can't run in multi queue mode, if nb_q_per_pool was set as 1. > Nb_q_per_pool is passed through to max_rx_q and max_tx_q in VF. > So if nb_q_per_pool is equal to 1, max_rx_q and max_tx_q can't be more > than 1 and VF multi queue mode will fail. > > Fixes: 27b609cbd1c6 (ethdev: move the multi-queue mode check) > > Signed-off-by: Yanglong Wu <yanglong.wu@intel.com> > --- > v1~v3: > coding style issue fix > --- > v4: > Rework according to comments > --- > drivers/net/ixgbe/ixgbe_ethdev.c | 2 -- > 1 file changed, 2 deletions(-) > > diff --git a/drivers/net/ixgbe/ixgbe_ethdev.c > b/drivers/net/ixgbe/ixgbe_ethdev.c > index baaeee5d9..6af4f8b2b 100644 > --- a/drivers/net/ixgbe/ixgbe_ethdev.c > +++ b/drivers/net/ixgbe/ixgbe_ethdev.c > @@ -2240,8 +2240,6 @@ ixgbe_check_mq_mode(struct rte_eth_dev *dev) > case ETH_MQ_RX_NONE: > /* if nothing mq mode configure, use default scheme */ > dev->data->dev_conf.rxmode.mq_mode = ETH_MQ_RX_VMDQ_ONLY; > - if (RTE_ETH_DEV_SRIOV(dev).nb_q_per_pool > 1) > - RTE_ETH_DEV_SRIOV(dev).nb_q_per_pool = 1; > break; > default: /* ETH_MQ_RX_DCB, ETH_MQ_RX_DCB_RSS or ETH_MQ_TX_DCB*/ > /* SRIOV only works in VMDq enable mode */ > -- > 2.11.0 ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [dpdk-dev] [PATCH v4] net/ixgbe: fix l3fwd start failed on VF 2018-01-11 15:29 ` Wu, Yanglong @ 2018-01-12 2:38 ` Wu, Yanglong 2018-01-12 3:03 ` Dai, Wei 0 siblings, 1 reply; 10+ messages in thread From: Wu, Yanglong @ 2018-01-12 2:38 UTC (permalink / raw) To: Dai, Wei, dev Hi After confirm with Daiwei and according to the data sheet, RTE_ETH_DEV_SRIOV(dev).nb_q_per_pool is the Max number of queue can be used in VF, that would be assigned as IXGBE_MAX_RX_QUEUE_NUM / RTE_ETH_DEV_SRIOV(dev).active, so that assigning nb_q_per_pool as 1 when PF is in ETH_MQ_RX_NONE, which will make VF can just use only 1 queue, is not right. So this patch is right. Yanglong -----Original Message----- From: Wu, Yanglong Sent: Thursday, January 11, 2018 11:29 PM To: Wu, Yanglong <yanglong.wu@intel.com>; Dai, Wei <wei.dai@intel.com>; dev@dpdk.org Subject: RE: [dpdk-dev] [PATCH v4] net/ixgbe: fix l3fwd start failed on VF Hi daiwei: After I double checked with this bug, the problem can be described as follow: When we start PF using testpmd like this: testpmd -c 0x3 -n 4 --socket-mem=1024,1024 --file-prefix=pf -w 0000:05:00.0 -w 0000:05:00.1 -- -i. The number of queue signed to each port would be 1 and the rxmode.mq_mode of ixgbe eth_dev would be set as ETH_MQ_RX_NONE. And this config info is make sense, so until now there is no problem, right? And according to the code: case ETH_MQ_RX_NONE: > /* if nothing mq mode configure, use default scheme */ > dev->data->dev_conf.rxmode.mq_mode = ETH_MQ_RX_VMDQ_ONLY; > - if (RTE_ETH_DEV_SRIOV(dev).nb_q_per_pool > 1) > - RTE_ETH_DEV_SRIOV(dev).nb_q_per_pool = 1; > break; The nb_q_per_pool of (RTE_ETH_DEV_SRIOV(dev).nb_q_per_pool would be set as 1, that means for VF the number of max queue can be used would be only 1. And next we start VF using l3fwd: l3fwd/build/l3fwd -c 0xc0000 -n 4 -w 0000:05:10.0 -w 0000:05:10.1 -- -p 0x3 --config '(0,0,18),(1,0,19)' --parse-ptype The number of rx_q equal to 1 and there is no problem. But, in l3fwd, the number of tx_q would equal to the number of core used in l3fwd, so here number of tx_q equal to 2 but as described above, the max number of queue can be used in VF is just only 1. So there is problem and l3fwd will check failed. So there is two ways to fix this bug, the first is using this patch, don't assign RTE_ETH_DEV_SRIOV(dev).nb_q_per_pool as 1 when mq_mode = ETH_MQ_RX_VMDQ_ONLY. And the second is that the number of tx_q would not assigned as the number of cores used in l3fwd. Which one do you think is make sense? Yanglong -----Original Message----- From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Wu, Yanglong Sent: Thursday, January 11, 2018 6:30 PM To: Dai, Wei <wei.dai@intel.com>; dev@dpdk.org Subject: Re: [dpdk-dev] [PATCH v4] net/ixgbe: fix l3fwd start failed on VF Yes, this problem is caused by PF passing a wrong number of queue to VF. So basically, I removed model check of "case ETH_MQ_RX_NONE". It seem right after I do that but I'am also not very sure the changing is really right. So thank to your comments, I will redebug this problem. -----Original Message----- From: Dai, Wei Sent: Thursday, January 11, 2018 5:48 PM To: Wu, Yanglong <yanglong.wu@intel.com>; dev@dpdk.org; Yang, Zhiyong <zhiyong.yang@intel.com> Cc: Wu, Yanglong <yanglong.wu@intel.com> Subject: RE: [dpdk-dev] [PATCH v4] net/ixgbe: fix l3fwd start failed on VF According to 82599 datasheet, VF working mode is controlled by its host PF. If VF is working in RSS or DCB mode, it can use more than 1 queues. That also means its host PF should enable RSS or DCB. The PF Rx mode can be ETH_MQ_RX_VMDQ_RSS or ETH_MQ_RX_VMDQ_DCB. So I think that DPDK app with PF cause this issue, not ixgbe PMD itself. By the way, you can reply your patch mail to provide more info about how You reproduce this issue to easy others' reviewing and testing your patch. Thanks > -----Original Message----- > From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Yanglong Wu > Sent: Tuesday, January 9, 2018 2:32 PM > To: dev@dpdk.org; Yang, Zhiyong <zhiyong.yang@intel.com> > Cc: Wu, Yanglong <yanglong.wu@intel.com> > Subject: [dpdk-dev] [PATCH v4] net/ixgbe: fix l3fwd start failed on VF > > VF can't run in multi queue mode, if nb_q_per_pool was set as 1. > Nb_q_per_pool is passed through to max_rx_q and max_tx_q in VF. > So if nb_q_per_pool is equal to 1, max_rx_q and max_tx_q can't be more > than 1 and VF multi queue mode will fail. > > Fixes: 27b609cbd1c6 (ethdev: move the multi-queue mode check) > > Signed-off-by: Yanglong Wu <yanglong.wu@intel.com> > --- > v1~v3: > coding style issue fix > --- > v4: > Rework according to comments > --- > drivers/net/ixgbe/ixgbe_ethdev.c | 2 -- > 1 file changed, 2 deletions(-) > > diff --git a/drivers/net/ixgbe/ixgbe_ethdev.c > b/drivers/net/ixgbe/ixgbe_ethdev.c > index baaeee5d9..6af4f8b2b 100644 > --- a/drivers/net/ixgbe/ixgbe_ethdev.c > +++ b/drivers/net/ixgbe/ixgbe_ethdev.c > @@ -2240,8 +2240,6 @@ ixgbe_check_mq_mode(struct rte_eth_dev *dev) > case ETH_MQ_RX_NONE: > /* if nothing mq mode configure, use default scheme */ > dev->data->dev_conf.rxmode.mq_mode = ETH_MQ_RX_VMDQ_ONLY; > - if (RTE_ETH_DEV_SRIOV(dev).nb_q_per_pool > 1) > - RTE_ETH_DEV_SRIOV(dev).nb_q_per_pool = 1; > break; > default: /* ETH_MQ_RX_DCB, ETH_MQ_RX_DCB_RSS or ETH_MQ_TX_DCB*/ > /* SRIOV only works in VMDq enable mode */ > -- > 2.11.0 ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [dpdk-dev] [PATCH v4] net/ixgbe: fix l3fwd start failed on VF 2018-01-12 2:38 ` Wu, Yanglong @ 2018-01-12 3:03 ` Dai, Wei 2018-01-13 3:30 ` Zhang, Helin 0 siblings, 1 reply; 10+ messages in thread From: Dai, Wei @ 2018-01-12 3:03 UTC (permalink / raw) To: Wu, Yanglong, dev > -----Original Message----- > From: Wu, Yanglong > Sent: Friday, January 12, 2018 10:38 AM > To: Dai, Wei <wei.dai@intel.com>; dev@dpdk.org > Subject: RE: [dpdk-dev] [PATCH v4] net/ixgbe: fix l3fwd start failed on VF > > Hi > After confirm with Daiwei and according to the data sheet, > RTE_ETH_DEV_SRIOV(dev).nb_q_per_pool is the Max number of queue can > be used in VF, that would be assigned as IXGBE_MAX_RX_QUEUE_NUM / > RTE_ETH_DEV_SRIOV(dev).active, so that assigning nb_q_per_pool as 1 > when PF is in ETH_MQ_RX_NONE, which will make VF can just use only 1 > queue, is not right. So this patch is right. > > Yanglong > > -----Original Message----- > From: Wu, Yanglong > Sent: Thursday, January 11, 2018 11:29 PM > To: Wu, Yanglong <yanglong.wu@intel.com>; Dai, Wei > <wei.dai@intel.com>; dev@dpdk.org > Subject: RE: [dpdk-dev] [PATCH v4] net/ixgbe: fix l3fwd start failed on VF > > Hi daiwei: > > After I double checked with this bug, the problem can be described as follow: > When we start PF using testpmd like this: testpmd -c 0x3 -n 4 > --socket-mem=1024,1024 --file-prefix=pf -w 0000:05:00.0 -w 0000:05:00.1 > -- -i. The number of queue signed to each port would be 1 and the > rxmode.mq_mode of ixgbe eth_dev would be set as ETH_MQ_RX_NONE. > And this config info is make sense, so until now there is no problem, right? > And according to the code: > case ETH_MQ_RX_NONE: > > /* if nothing mq mode configure, use default scheme */ > > dev->data->dev_conf.rxmode.mq_mode = > ETH_MQ_RX_VMDQ_ONLY; > > - if (RTE_ETH_DEV_SRIOV(dev).nb_q_per_pool > 1) > > - RTE_ETH_DEV_SRIOV(dev).nb_q_per_pool = 1; > > break; > The nb_q_per_pool of (RTE_ETH_DEV_SRIOV(dev).nb_q_per_pool would > be set as 1, that means for VF the number of max queue can be used would > be only 1. > > And next we start VF using l3fwd: > l3fwd/build/l3fwd -c 0xc0000 -n 4 -w 0000:05:10.0 -w 0000:05:10.1 -- -p 0x3 > --config '(0,0,18),(1,0,19)' --parse-ptype The number of rx_q equal to 1 and > there is no problem. But, in l3fwd, the number of tx_q would equal to the > number of core used in l3fwd, so here number of tx_q equal to 2 but as > described above, the max number of queue can be used in VF is just only 1. > So there is problem and l3fwd will check failed. > > > So there is two ways to fix this bug, the first is using this patch, don't assign > RTE_ETH_DEV_SRIOV(dev).nb_q_per_pool as 1 when mq_mode = > ETH_MQ_RX_VMDQ_ONLY. > And the second is that the number of tx_q would not assigned as the number > of cores used in l3fwd. > > Which one do you think is make sense? > > > > > Yanglong > > > -----Original Message----- > From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Wu, Yanglong > Sent: Thursday, January 11, 2018 6:30 PM > To: Dai, Wei <wei.dai@intel.com>; dev@dpdk.org > Subject: Re: [dpdk-dev] [PATCH v4] net/ixgbe: fix l3fwd start failed on VF > > Yes, this problem is caused by PF passing a wrong number of queue to VF. So > basically, I removed model check of "case ETH_MQ_RX_NONE". It seem > right after I do that but I'am also not very sure the changing is really right. So > thank to your comments, I will redebug this problem. > > -----Original Message----- > From: Dai, Wei > Sent: Thursday, January 11, 2018 5:48 PM > To: Wu, Yanglong <yanglong.wu@intel.com>; dev@dpdk.org; Yang, Zhiyong > <zhiyong.yang@intel.com> > Cc: Wu, Yanglong <yanglong.wu@intel.com> > Subject: RE: [dpdk-dev] [PATCH v4] net/ixgbe: fix l3fwd start failed on VF > > According to 82599 datasheet, VF working mode is controlled by its host PF. > If VF is working in RSS or DCB mode, it can use more than 1 queues. > That also means its host PF should enable RSS or DCB. > The PF Rx mode can be ETH_MQ_RX_VMDQ_RSS or > ETH_MQ_RX_VMDQ_DCB. > So I think that DPDK app with PF cause this issue, not ixgbe PMD itself. > > By the way, you can reply your patch mail to provide more info about how > You reproduce this issue to easy others' reviewing and testing your patch. > > Thanks > > > -----Original Message----- > > From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Yanglong Wu > > Sent: Tuesday, January 9, 2018 2:32 PM > > To: dev@dpdk.org; Yang, Zhiyong <zhiyong.yang@intel.com> > > Cc: Wu, Yanglong <yanglong.wu@intel.com> > > Subject: [dpdk-dev] [PATCH v4] net/ixgbe: fix l3fwd start failed on VF > > > > VF can't run in multi queue mode, if nb_q_per_pool was set as 1. > > Nb_q_per_pool is passed through to max_rx_q and max_tx_q in VF. > > So if nb_q_per_pool is equal to 1, max_rx_q and max_tx_q can't be more > > than 1 and VF multi queue mode will fail. > > > > Fixes: 27b609cbd1c6 (ethdev: move the multi-queue mode check) > > > > Signed-off-by: Yanglong Wu <yanglong.wu@intel.com> Acked-by: Wei Dai <wei.dai@intel.com> > > --- > > v1~v3: > > coding style issue fix > > --- > > v4: > > Rework according to comments > > --- > > drivers/net/ixgbe/ixgbe_ethdev.c | 2 -- > > 1 file changed, 2 deletions(-) > > > > diff --git a/drivers/net/ixgbe/ixgbe_ethdev.c > > b/drivers/net/ixgbe/ixgbe_ethdev.c > > index baaeee5d9..6af4f8b2b 100644 > > --- a/drivers/net/ixgbe/ixgbe_ethdev.c > > +++ b/drivers/net/ixgbe/ixgbe_ethdev.c > > @@ -2240,8 +2240,6 @@ ixgbe_check_mq_mode(struct rte_eth_dev > *dev) > > case ETH_MQ_RX_NONE: > > /* if nothing mq mode configure, use default scheme */ > > dev->data->dev_conf.rxmode.mq_mode = > ETH_MQ_RX_VMDQ_ONLY; > > - if (RTE_ETH_DEV_SRIOV(dev).nb_q_per_pool > 1) > > - RTE_ETH_DEV_SRIOV(dev).nb_q_per_pool = 1; > > break; > > default: /* ETH_MQ_RX_DCB, ETH_MQ_RX_DCB_RSS or > ETH_MQ_TX_DCB*/ > > /* SRIOV only works in VMDq enable mode */ > > -- > > 2.11.0 ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [dpdk-dev] [PATCH v4] net/ixgbe: fix l3fwd start failed on VF 2018-01-12 3:03 ` Dai, Wei @ 2018-01-13 3:30 ` Zhang, Helin 0 siblings, 0 replies; 10+ messages in thread From: Zhang, Helin @ 2018-01-13 3:30 UTC (permalink / raw) To: Dai, Wei, Wu, Yanglong, dev > -----Original Message----- > From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Dai, Wei > Sent: Friday, January 12, 2018 11:04 AM > To: Wu, Yanglong; dev@dpdk.org > Subject: Re: [dpdk-dev] [PATCH v4] net/ixgbe: fix l3fwd start failed on VF > > > -----Original Message----- > > From: Wu, Yanglong > > Sent: Friday, January 12, 2018 10:38 AM > > To: Dai, Wei <wei.dai@intel.com>; dev@dpdk.org > > Subject: RE: [dpdk-dev] [PATCH v4] net/ixgbe: fix l3fwd start failed > > on VF > > > > Hi > > After confirm with Daiwei and according to the data sheet, > > RTE_ETH_DEV_SRIOV(dev).nb_q_per_pool is the Max number of queue can > be > > used in VF, that would be assigned as IXGBE_MAX_RX_QUEUE_NUM / > > RTE_ETH_DEV_SRIOV(dev).active, so that assigning nb_q_per_pool as 1 > > when PF is in ETH_MQ_RX_NONE, which will make VF can just use only 1 > > queue, is not right. So this patch is right. > > > > Yanglong > > > > -----Original Message----- > > From: Wu, Yanglong > > Sent: Thursday, January 11, 2018 11:29 PM > > To: Wu, Yanglong <yanglong.wu@intel.com>; Dai, Wei > > <wei.dai@intel.com>; dev@dpdk.org > > Subject: RE: [dpdk-dev] [PATCH v4] net/ixgbe: fix l3fwd start failed > > on VF > > > > Hi daiwei: > > > > After I double checked with this bug, the problem can be described as follow: > > When we start PF using testpmd like this: testpmd -c 0x3 -n 4 > > --socket-mem=1024,1024 --file-prefix=pf -w 0000:05:00.0 -w > > 0000:05:00.1 > > -- -i. The number of queue signed to each port would be 1 and the > > rxmode.mq_mode of ixgbe eth_dev would be set as ETH_MQ_RX_NONE. > > And this config info is make sense, so until now there is no problem, right? > > And according to the code: > > case ETH_MQ_RX_NONE: > > > /* if nothing mq mode configure, use default scheme > */ > > > dev->data->dev_conf.rxmode.mq_mode = > > ETH_MQ_RX_VMDQ_ONLY; > > > - if (RTE_ETH_DEV_SRIOV(dev).nb_q_per_pool > 1) > > > - RTE_ETH_DEV_SRIOV(dev).nb_q_per_pool = 1; > > > break; > > The nb_q_per_pool of (RTE_ETH_DEV_SRIOV(dev).nb_q_per_pool would be > > set as 1, that means for VF the number of max queue can be used would > > be only 1. > > > > And next we start VF using l3fwd: > > l3fwd/build/l3fwd -c 0xc0000 -n 4 -w 0000:05:10.0 -w 0000:05:10.1 -- > > -p 0x3 --config '(0,0,18),(1,0,19)' --parse-ptype The number of rx_q > > equal to 1 and there is no problem. But, in l3fwd, the number of tx_q > > would equal to the number of core used in l3fwd, so here number of > > tx_q equal to 2 but as described above, the max number of queue can be > used in VF is just only 1. > > So there is problem and l3fwd will check failed. > > > > > > So there is two ways to fix this bug, the first is using this patch, > > don't assign RTE_ETH_DEV_SRIOV(dev).nb_q_per_pool as 1 when mq_mode > = > > ETH_MQ_RX_VMDQ_ONLY. > > And the second is that the number of tx_q would not assigned as the > > number of cores used in l3fwd. > > > > Which one do you think is make sense? > > > > > > > > > > Yanglong > > > > > > -----Original Message----- > > From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Wu, Yanglong > > Sent: Thursday, January 11, 2018 6:30 PM > > To: Dai, Wei <wei.dai@intel.com>; dev@dpdk.org > > Subject: Re: [dpdk-dev] [PATCH v4] net/ixgbe: fix l3fwd start failed > > on VF > > > > Yes, this problem is caused by PF passing a wrong number of queue to > > VF. So basically, I removed model check of "case ETH_MQ_RX_NONE". It > > seem right after I do that but I'am also not very sure the changing is > > really right. So thank to your comments, I will redebug this problem. > > > > -----Original Message----- > > From: Dai, Wei > > Sent: Thursday, January 11, 2018 5:48 PM > > To: Wu, Yanglong <yanglong.wu@intel.com>; dev@dpdk.org; Yang, Zhiyong > > <zhiyong.yang@intel.com> > > Cc: Wu, Yanglong <yanglong.wu@intel.com> > > Subject: RE: [dpdk-dev] [PATCH v4] net/ixgbe: fix l3fwd start failed > > on VF > > > > According to 82599 datasheet, VF working mode is controlled by its host PF. > > If VF is working in RSS or DCB mode, it can use more than 1 queues. > > That also means its host PF should enable RSS or DCB. > > The PF Rx mode can be ETH_MQ_RX_VMDQ_RSS or > ETH_MQ_RX_VMDQ_DCB. > > So I think that DPDK app with PF cause this issue, not ixgbe PMD itself. > > > > By the way, you can reply your patch mail to provide more info about > > how You reproduce this issue to easy others' reviewing and testing your > patch. > > > > Thanks > > > > > -----Original Message----- > > > From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Yanglong Wu > > > Sent: Tuesday, January 9, 2018 2:32 PM > > > To: dev@dpdk.org; Yang, Zhiyong <zhiyong.yang@intel.com> > > > Cc: Wu, Yanglong <yanglong.wu@intel.com> > > > Subject: [dpdk-dev] [PATCH v4] net/ixgbe: fix l3fwd start failed on > > > VF > > > > > > VF can't run in multi queue mode, if nb_q_per_pool was set as 1. > > > Nb_q_per_pool is passed through to max_rx_q and max_tx_q in VF. > > > So if nb_q_per_pool is equal to 1, max_rx_q and max_tx_q can't be > > > more than 1 and VF multi queue mode will fail. > > > > > > Fixes: 27b609cbd1c6 (ethdev: move the multi-queue mode check) > > > > > > Signed-off-by: Yanglong Wu <yanglong.wu@intel.com> > Acked-by: Wei Dai <wei.dai@intel.com> Applied to dpdk-next-net-intel, with minor commit log changes. Thanks! /Helin ^ permalink raw reply [flat|nested] 10+ messages in thread
end of thread, other threads:[~2018-01-13 3:30 UTC | newest]
Thread overview: 10+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
[not found] <20171102161031.167892-1-yanglong.wu@intel.com>
2017-11-20 2:00 ` [dpdk-dev] [PATCH v2] net/ixgbe: fix l3fwd start failed on VF Yanglong Wu
2017-11-20 2:25 ` [dpdk-dev] [PATCH v3] " Yanglong Wu
2017-11-20 2:37 ` Yang, Zhiyong
2018-01-09 6:32 ` [dpdk-dev] [PATCH v4] " Yanglong Wu
2018-01-11 9:48 ` Dai, Wei
2018-01-11 10:29 ` Wu, Yanglong
2018-01-11 15:29 ` Wu, Yanglong
2018-01-12 2:38 ` Wu, Yanglong
2018-01-12 3:03 ` Dai, Wei
2018-01-13 3:30 ` Zhang, Helin
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).