[dpdk-dev] IGB_UIO port unbinding

[dpdk-dev] IGB_UIO port unbinding

[Burakov, Anatoly]

Hi everyone

As you may or may not know, the CONFIG_RTE_EAL_UNBIND_PORTS was deprecated in the official Intel(r) DPDK 1.6.0 release package. However, the code itself (e.g. in lib/librte_eal/linuxapp/eal_pci.c and other places) to support that config option was not removed. My question would be, would anyone be opposed to a patch to remove that code entirely? E.g. does anyone use that functionality?

Best regards,
Anatoly Burakov
DPDK SW Engineer

--------------------------------------------------------------
Intel Shannon Limited
Registered in Ireland
Registered Office: Collinstown Industrial Park, Leixlip, County Kildare
Registered Number: 308263
Business address: Dromore House, East Park, Shannon, Co. Clare

Re: [dpdk-dev] IGB_UIO port unbinding

[Thomas Monjalon]

Hi,

2014-04-10 09:39, Burakov, Anatoly:
> As you may or may not know, the CONFIG_RTE_EAL_UNBIND_PORTS was deprecated
> in the official Intel(r) DPDK 1.6.0 release package. However, the code
> itself (e.g. in lib/librte_eal/linuxapp/eal_pci.c and other places) to
> support that config option was not removed. My question would be, would
> anyone be opposed to a patch to remove that code entirely? E.g. does anyone
> use that functionality?

On my side, it's OK to remove it.
I think you can prepare the patch and we'll wait 1 week before applying it.
If nobody complains, this feature will be definitely removed.

-- 
Thomas

Re: [dpdk-dev] IGB_UIO port unbinding

[Prashant Upadhyaya]

Hi,

There was a usecase with ESXi VMXNET3 NIC where I had to use this parameter set to Y to make it work.
So kindly ensure that the initialization of vmxnet3 NIC is not vulnerable.

Regards
-Prashant


-----Original Message-----
From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Thomas Monjalon
Sent: Thursday, April 10, 2014 5:10 PM
To: Burakov, Anatoly
Cc: dev@dpdk.org
Subject: Re: [dpdk-dev] IGB_UIO port unbinding

Hi,

2014-04-10 09:39, Burakov, Anatoly:
> As you may or may not know, the CONFIG_RTE_EAL_UNBIND_PORTS was
> deprecated in the official Intel(r) DPDK 1.6.0 release package.
> However, the code itself (e.g. in lib/librte_eal/linuxapp/eal_pci.c
> and other places) to support that config option was not removed. My
> question would be, would anyone be opposed to a patch to remove that
> code entirely? E.g. does anyone use that functionality?

On my side, it's OK to remove it.
I think you can prepare the patch and we'll wait 1 week before applying it.
If nobody complains, this feature will be definitely removed.

--
Thomas


"DISCLAIMER: This message is proprietary to Aricent and is intended solely for the use of the individual to whom it is addressed. It may contain privileged or confidential information and should not be circulated or used for any purpose other than for what it is intended. If you have received this message in error, please notify the originator immediately. If you are not the intended recipient, you are notified that you are strictly prohibited from using, copying, altering, or disclosing the contents of this message. Aricent accepts no responsibility for loss or damage arising from the use of the information transmitted by this email including damage from virus."

Re: [dpdk-dev] IGB_UIO port unbinding

[Thomas Monjalon]

Hi,

2014-04-10 21:34, Prashant Upadhyaya:
> There was a usecase with ESXi VMXNET3 NIC where I had to use this parameter
> set to Y to make it work. So kindly ensure that the initialization of
> vmxnet3 NIC is not vulnerable.

This issue is fixed by this commit:
	http://dpdk.org/browse/dpdk/commit/?id=18f02ff75949de9c2468

So it's not a blocker for removing RTE_EAL_UNBIND_PORTS.

-- 
Thomas

Re: [dpdk-dev] IGB_UIO port unbinding

[Burakov, Anatoly]

Hi Prashant,

> There was a usecase with ESXi VMXNET3 NIC where I had to use this
> parameter set to Y to make it work.
> So kindly ensure that the initialization of vmxnet3 NIC is not vulnerable.
> 

Did you try it on the latest 1.6.x code (that option was removed)?

More to the point, I can't see how this would be relevant to vmxnet3 initialization. All this config option does is automatically unbinds NICs for you, which is something you can do manually with the included igb_uio_bind.py/pci_unbind.py script anyway. Could you please elaborate on how exactly this option helped with vmxnet3 initialization?

Best regards,
Anatoly Burakov
DPDK SW Engineer

--------------------------------------------------------------
Intel Shannon Limited
Registered in Ireland
Registered Office: Collinstown Industrial Park, Leixlip, County Kildare
Registered Number: 308263
Business address: Dromore House, East Park, Shannon, Co. Clare

Re: [dpdk-dev] IGB_UIO port unbinding

[Prashant Upadhyaya]

Hi Anatoly,

I might have used the term 'initialization' in a wrong fashion.
But I have confirmed, the issue was related to this commit (which Thomas brought to my notice) --

http://dpdk.org/browse/dpdk/commit/?id=18f02ff75949de9c2468

The above should get you the context for the original issue.

Regards
-Prashant

-----Original Message-----
From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Burakov, Anatoly
Sent: Friday, April 11, 2014 2:29 PM
To: dev@dpdk.org
Subject: Re: [dpdk-dev] IGB_UIO port unbinding

Hi Prashant,

> There was a usecase with ESXi VMXNET3 NIC where I had to use this
> parameter set to Y to make it work.
> So kindly ensure that the initialization of vmxnet3 NIC is not vulnerable.
>

Did you try it on the latest 1.6.x code (that option was removed)?

More to the point, I can't see how this would be relevant to vmxnet3 initialization. All this config option does is automatically unbinds NICs for you, which is something you can do manually with the included igb_uio_bind.py/pci_unbind.py script anyway. Could you please elaborate on how exactly this option helped with vmxnet3 initialization?

Best regards,
Anatoly Burakov
DPDK SW Engineer

--------------------------------------------------------------
Intel Shannon Limited
Registered in Ireland
Registered Office: Collinstown Industrial Park, Leixlip, County Kildare Registered Number: 308263 Business address: Dromore House, East Park, Shannon, Co. Clare





"DISCLAIMER: This message is proprietary to Aricent and is intended solely for the use of the individual to whom it is addressed. It may contain privileged or confidential information and should not be circulated or used for any purpose other than for what it is intended. If you have received this message in error, please notify the originator immediately. If you are not the intended recipient, you are notified that you are strictly prohibited from using, copying, altering, or disclosing the contents of this message. Aricent accepts no responsibility for loss or damage arising from the use of the information transmitted by this email including damage from virus."

Re: [dpdk-dev] IGB_UIO port unbinding

[Burakov, Anatoly]

Hi Prashant,

> I might have used the term 'initialization' in a wrong fashion.
> But I have confirmed, the issue was related to this commit (which Thomas
> brought to my notice) --
> 
> http://dpdk.org/browse/dpdk/commit/?id=18f02ff75949de9c2468
> 
> The above should get you the context for the original issue.

Thanks, that gives me some context. I'm not familiar enough with dpdk.org codebase, so I wasn't aware of these issues. However, from the looks of it, I doubt that vmxnet3 initialization would be affected - the RTE_PCI_DRV_NEED_IGB_UIO flag will be set regardless (in fact, the commit you linked to has removed dependency on RTE_EAL_UNBIND_PORTS config option for vmxnet3), and vmxnet3 should initialize correctly just like the rest of the NICs with or without that config option. I am open to correction on that one however :-)

Best regards,
Anatoly Burakov
DPDK SW Engineer

--------------------------------------------------------------
Intel Shannon Limited
Registered in Ireland
Registered Office: Collinstown Industrial Park, Leixlip, County Kildare
Registered Number: 308263
Business address: Dromore House, East Park, Shannon, Co. Clare

Re: [dpdk-dev] IGB_UIO port unbinding

[Stephen Hemminger]

On Thu, 10 Apr 2014 09:39:35 +0000
"Burakov, Anatoly" <anatoly.burakov@intel.com> wrote:

> Hi everyone
> 
> As you may or may not know, the CONFIG_RTE_EAL_UNBIND_PORTS was deprecated in the official Intel(r) DPDK 1.6.0 release package. However, the code itself (e.g. in lib/librte_eal/linuxapp/eal_pci.c and other places) to support that config option was not removed. My question would be, would anyone be opposed to a patch to remove that code entirely? E.g. does anyone use that functionality?
> 
> Best regards,
> Anatoly Burakov
> DPDK SW Engineer

I actually liked the unbind code that was there. It meant that additional logic
did not have to be added to the startup script to do PCI discovery and finding/mapping
devices.

As it is the EAL startup process is a bit of a mess. It assumes that various
bits of information and infrastructure are already in place prior to starting
the DPDK application.

Other silly DPDK environment issues are: why does DPDK have to know the number
of memory channels in the system? How is the end user of an application supposed
to know this? Sure your hacker might but not end user. It is difficult to impossible
to determine this information at runtime. The best I could find is to use DMI decode
to count number of memory banks and guess!

The CPU mask is also something that should "do the right thing" and use all CPU's
unless told otherwise. Having to wrap every DPDK application in a shell script
makes the application more brittle and creates overlapping effort.

Re: [dpdk-dev] IGB_UIO port unbinding

[Burakov, Anatoly]

Hi Stephen,

> I actually liked the unbind code that was there. It meant that additional logic
> did not have to be added to the startup script to do PCI discovery and
> finding/mapping devices.
> 
> As it is the EAL startup process is a bit of a mess. It assumes that various bits
> of information and infrastructure are already in place prior to starting the
> DPDK application.

I would argue that it's not the responsibility of the application to set up its environment. The EAL startup process is indeed a bit all over the place, but the alternative would be worse. Those "various bits of information and infrastructure" you're referring to include, say, hugepages - I don't think it's a good idea for an application to meddle with system's hugepages any more than I think it's a good idea for it to meddle with the driver bindings.

Plus, current state of affairs makes for headaches in the future when support for other drivers is implemented (say, VFIO). It's much better to move the responsibility of binding the drivers to the user, and DPDK provides quite convenient means to do that (the PCI binding Python script) and also a centralized script to prepare the DPDK environment without much hassle (the setup.sh).

Best regards,
Anatoly Burakov
DPDK SW Engineer

--------------------------------------------------------------
Intel Shannon Limited
Registered in Ireland
Registered Office: Collinstown Industrial Park, Leixlip, County Kildare
Registered Number: 308263
Business address: Dromore House, East Park, Shannon, Co. Clare

Re: [dpdk-dev] IGB_UIO port unbinding

[Stephen Hemminger]

On Mon, 14 Apr 2014 08:57:54 +0000
"Burakov, Anatoly" <anatoly.burakov@intel.com> wrote:

> Hi Stephen,
> 
> > I actually liked the unbind code that was there. It meant that additional logic
> > did not have to be added to the startup script to do PCI discovery and
> > finding/mapping devices.
> > 
> > As it is the EAL startup process is a bit of a mess. It assumes that various bits
> > of information and infrastructure are already in place prior to starting the
> > DPDK application.
> 
> I would argue that it's not the responsibility of the application to set up its environment. The EAL startup process is indeed a bit all over the place, but the alternative would be worse. Those "various bits of information and infrastructure" you're referring to include, say, hugepages - I don't think it's a good idea for an application to meddle with system's hugepages any more than I think it's a good idea for it to meddle with the driver bindings.
> 
> Plus, current state of affairs makes for headaches in the future when support for other drivers is implemented (say, VFIO). It's much better to move the responsibility of binding the drivers to the user, and DPDK provides quite convenient means to do that (the PCI binding Python script) and also a centralized script to prepare the DPDK environment without much hassle (the setup.sh).
> 
> Best regards,
> Anatoly Burakov
> DPDK SW Engineer
> 

The problem is that doing the initialization requires duplicating all the
logic to probe the PCI bus and do blacklisting etc in the startup application.

Re: [dpdk-dev] IGB_UIO port unbinding

[Burakov, Anatoly]

Hi Stephen,
 
> The problem is that doing the initialization requires duplicating all the logic to
> probe the PCI bus and do blacklisting etc in the startup application.

Not quite sure what you're referring to here, could you please elaborate a bit more? If you mean the EAL PCI probing and blacklisting/whitelisting, then yes - we have to do it, but this is out of necessity more than anything else - there's no way around that since DPDK is using userspace drivers. DPDK doesn't require to bind/unbind kernel drivers in the EAL however - there is already a way to do that that doesn't involve EAL-side logic. EAL simply checks if a particular PCI device is bound to a certain driver and if it's blacklisted (in other words, EAL checks if it is able to use the device, but doesn't try to modify the environment).

Best regards,
Anatoly Burakov
DPDK SW Engineer

--------------------------------------------------------------
Intel Shannon Limited
Registered in Ireland
Registered Office: Collinstown Industrial Park, Leixlip, County Kildare
Registered Number: 308263
Business address: Dromore House, East Park, Shannon, Co. Clare