[dpdk-dev] nic loopback

[dpdk-dev] nic loopback

[Alex Markuze]

Hi,
I'm trying to send packets from an application to it self, meaning smac  ==
dmac.
I'm working with intel 82599 virtual function. But it seems that these
packets are lost.

Is there a software/hw limitation I'm missing here (some additional
anti-spoofing)? AFAIK modern NICs with sriov are mini switches so the hw
loopback should work, at least thats the theory.


Thanks.

Re: [dpdk-dev] nic loopback

[Liang, Cunming]

> -----Original Message-----
> From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Alex Markuze
> Sent: Tuesday, October 21, 2014 12:24 AM
> To: dev@dpdk.org
> Subject: [dpdk-dev] nic loopback
> 
> Hi,
> I'm trying to send packets from an application to it self, meaning smac  ==
> dmac.
> I'm working with intel 82599 virtual function. But it seems that these
> packets are lost.
> 
> Is there a software/hw limitation I'm missing here (some additional
> anti-spoofing)? AFAIK modern NICs with sriov are mini switches so the hw
> loopback should work, at least thats the theory.
> 
[Liang, Cunming] You could have a check on register LLE(PFVMTXSW[n]).
Which allow an individual pool to be able to send traffic and have it loopback to itself.
> 
> Thanks.

Re: [dpdk-dev] nic loopback

[Alex Markuze]

How can I set/query this bit (LLE(PFVMTXSW[n]), intel 82599 ) on ESX, or
any other friendlier environment like Linux?

On Tue, Oct 21, 2014 at 4:18 AM, Liang, Cunming <cunming.liang@intel.com>
wrote:

>
>
> > -----Original Message-----
> > From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Alex Markuze
> > Sent: Tuesday, October 21, 2014 12:24 AM
> > To: dev@dpdk.org
> > Subject: [dpdk-dev] nic loopback
> >
> > Hi,
> > I'm trying to send packets from an application to it self, meaning smac
> ==
> > dmac.
> > I'm working with intel 82599 virtual function. But it seems that these
> > packets are lost.
> >
> > Is there a software/hw limitation I'm missing here (some additional
> > anti-spoofing)? AFAIK modern NICs with sriov are mini switches so the hw
> > loopback should work, at least thats the theory.
> >
> [Liang, Cunming] You could have a check on register LLE(PFVMTXSW[n]).
> Which allow an individual pool to be able to send traffic and have it
> loopback to itself.
> >
> > Thanks.
>

Re: [dpdk-dev] nic loopback

[Liang, Cunming]

It’s a pain VF can’t set the register directly.
As kernel ixgbe don’t support to set the value, I’m afraid you have to modify kernel ixgbe.
If your purpose is mainly for testing purpose.
One option is you can just set the register bit value to full 1 during device initialization.
Another option is you can choose to use DPDK as host PF.
Running testpmd in host, and set such register by interactive command line.

Ideally it’s better to add a kind of VF to PF mailbox message.
Host PF delegate VF to enable the local pool loopback.
So during runtime, VF can proactive to enable/disable the ability.


From: Alex Markuze [mailto:alex@weka.io]
Sent: Tuesday, October 21, 2014 11:16 PM
To: Liang, Cunming
Cc: dev@dpdk.org
Subject: Re: [dpdk-dev] nic loopback

How can I set/query this bit (LLE(PFVMTXSW[n]), intel 82599 ) on ESX, or any other friendlier environment like Linux?

On Tue, Oct 21, 2014 at 4:18 AM, Liang, Cunming <cunming.liang@intel.com<mailto:cunming.liang@intel.com>> wrote:


> -----Original Message-----
> From: dev [mailto:dev-bounces@dpdk.org<mailto:dev-bounces@dpdk.org>] On Behalf Of Alex Markuze
> Sent: Tuesday, October 21, 2014 12:24 AM
> To: dev@dpdk.org<mailto:dev@dpdk.org>
> Subject: [dpdk-dev] nic loopback
>
> Hi,
> I'm trying to send packets from an application to it self, meaning smac  ==
> dmac.
> I'm working with intel 82599 virtual function. But it seems that these
> packets are lost.
>
> Is there a software/hw limitation I'm missing here (some additional
> anti-spoofing)? AFAIK modern NICs with sriov are mini switches so the hw
> loopback should work, at least thats the theory.
>
[Liang, Cunming] You could have a check on register LLE(PFVMTXSW[n]).
Which allow an individual pool to be able to send traffic and have it loopback to itself.
>
> Thanks.

[dpdk-dev] FW: nic loopback

[Zhu, Heqing]

One line comment inline. 

> -----Original Message-----
> From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Liang, Cunming
> Sent: Tuesday, October 21, 2014 8:33 PM
> To: Alex Markuze
> Cc: dev@dpdk.org
> Subject: Re: [dpdk-dev] nic loopback
> 
> It’s a pain VF can’t set the register directly.
> As kernel ixgbe don’t support to set the value, I’m afraid you have to modify
> kernel ixgbe.
> If your purpose is mainly for testing purpose.
> One option is you can just set the register bit value to full 1 during device
> initialization.
> Another option is you can choose to use DPDK as host PF.
> Running testpmd in host, and set such register by interactive command line.
> 
> Ideally it’s better to add a kind of VF to PF mailbox message.
> Host PF delegate VF to enable the local pool loopback.
> So during runtime, VF can proactive to enable/disable the ability.

[heqing] Such a proposal has been discussed a few times, but the kernel driver does not accept this due to the security concern.

> 
> 
> From: Alex Markuze [mailto:alex@weka.io]
> Sent: Tuesday, October 21, 2014 11:16 PM
> To: Liang, Cunming
> Cc: dev@dpdk.org
> Subject: Re: [dpdk-dev] nic loopback
> 
> How can I set/query this bit (LLE(PFVMTXSW[n]), intel 82599 ) on ESX, or any
> other friendlier environment like Linux?
> 
> On Tue, Oct 21, 2014 at 4:18 AM, Liang, Cunming
> <cunming.liang@intel.com<mailto:cunming.liang@intel.com>> wrote:
> 
> 
> > -----Original Message-----
> > From: dev
> [mailto:dev-bounces@dpdk.org<mailto:dev-bounces@dpdk.org>]
> > On Behalf Of Alex Markuze
> > Sent: Tuesday, October 21, 2014 12:24 AM
> > To: dev@dpdk.org<mailto:dev@dpdk.org>
> > Subject: [dpdk-dev] nic loopback
> >
> > Hi,
> > I'm trying to send packets from an application to it self, meaning
> > smac  == dmac.
> > I'm working with intel 82599 virtual function. But it seems that these
> > packets are lost.
> >
> > Is there a software/hw limitation I'm missing here (some additional
> > anti-spoofing)? AFAIK modern NICs with sriov are mini switches so the
> > hw loopback should work, at least thats the theory.
> >
> [Liang, Cunming] You could have a check on register LLE(PFVMTXSW[n]).
> Which allow an individual pool to be able to send traffic and have it loopback
> to itself.
> >
> > Thanks.

Re: [dpdk-dev] FW: nic loopback

[alex]

On Wed, Oct 22, 2014 at 7:37 AM, Zhu, Heqing <heqing.zhu@intel.com> wrote:

> One line comment inline.
>
> > -----Original Message-----
> > From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Liang, Cunming
> > Sent: Tuesday, October 21, 2014 8:33 PM
> > To: Alex Markuze
> > Cc: dev@dpdk.org
> > Subject: Re: [dpdk-dev] nic loopback
> >
> > It’s a pain VF can’t set the register directly.
> > As kernel ixgbe don’t support to set the value, I’m afraid you have to
> modify
> > kernel ixgbe.
> > If your purpose is mainly for testing purpose.
> > One option is you can just set the register bit value to full 1 during
> device
> > initialization.
> > Another option is you can choose to use DPDK as host PF.
> > Running testpmd in host, and set such register by interactive command
> line.
> >
> > Ideally it’s better to add a kind of VF to PF mailbox message.
> > Host PF delegate VF to enable the local pool loopback.
> > So during runtime, VF can proactive to enable/disable the ability.
>
> [heqing] Such a proposal has been discussed a few times, but the kernel
> driver does not accept this due to the security concern.


I will try a different approach, Is there a tool available by intel for
82599 nics that can access the NIC's configuration and modify these
registers manually? w/o Modifying hypervisor drivers and/or using PF?

>
> >
> >
> > From: Alex Markuze [mailto:alex@weka.io]
> > Sent: Tuesday, October 21, 2014 11:16 PM
> > To: Liang, Cunming
> > Cc: dev@dpdk.org
> > Subject: Re: [dpdk-dev] nic loopback
> >
> > How can I set/query this bit (LLE(PFVMTXSW[n]), intel 82599 ) on ESX, or
> any
> > other friendlier environment like Linux?
> >
> > On Tue, Oct 21, 2014 at 4:18 AM, Liang, Cunming
> > <cunming.liang@intel.com<mailto:cunming.liang@intel.com>> wrote:
> >
> >
> > > -----Original Message-----
> > > From: dev
> > [mailto:dev-bounces@dpdk.org<mailto:dev-bounces@dpdk.org>]
> > > On Behalf Of Alex Markuze
> > > Sent: Tuesday, October 21, 2014 12:24 AM
> > > To: dev@dpdk.org<mailto:dev@dpdk.org>
> > > Subject: [dpdk-dev] nic loopback
> > >
> > > Hi,
> > > I'm trying to send packets from an application to it self, meaning
> > > smac  == dmac.
> > > I'm working with intel 82599 virtual function. But it seems that these
> > > packets are lost.
> > >
> > > Is there a software/hw limitation I'm missing here (some additional
> > > anti-spoofing)? AFAIK modern NICs with sriov are mini switches so the
> > > hw loopback should work, at least thats the theory.
> > >
> > [Liang, Cunming] You could have a check on register LLE(PFVMTXSW[n]).
> > Which allow an individual pool to be able to send traffic and have it
> loopback
> > to itself.
> > >
> > > Thanks.
>
>

Re: [dpdk-dev] FW: nic loopback

[Liang, Cunming]

From: alex [mailto:alex@weka.io]
Sent: Wednesday, October 22, 2014 3:42 PM
To: Zhu, Heqing
Cc: Liang, Cunming; dev@dpdk.org
Subject: Re: FW: [dpdk-dev] nic loopback



On Wed, Oct 22, 2014 at 7:37 AM, Zhu, Heqing <heqing.zhu@intel.com<mailto:heqing.zhu@intel.com>> wrote:
One line comment inline.

> -----Original Message-----
> From: dev [mailto:dev-bounces@dpdk.org<mailto:dev-bounces@dpdk.org>] On Behalf Of Liang, Cunming
> Sent: Tuesday, October 21, 2014 8:33 PM
> To: Alex Markuze
> Cc: dev@dpdk.org<mailto:dev@dpdk.org>
> Subject: Re: [dpdk-dev] nic loopback
>
> It’s a pain VF can’t set the register directly.
> As kernel ixgbe don’t support to set the value, I’m afraid you have to modify
> kernel ixgbe.
> If your purpose is mainly for testing purpose.
> One option is you can just set the register bit value to full 1 during device
> initialization.
> Another option is you can choose to use DPDK as host PF.
> Running testpmd in host, and set such register by interactive command line.
>
> Ideally it’s better to add a kind of VF to PF mailbox message.
> Host PF delegate VF to enable the local pool loopback.
> So during runtime, VF can proactive to enable/disable the ability.

[heqing] Such a proposal has been discussed a few times, but the kernel driver does not accept this due to the security concern.

I will try a different approach, Is there a tool available by intel for 82599 nics that can access the NIC's configuration and modify these registers manually? w/o Modifying hypervisor drivers and/or using PF?
[Liang, Cunming] I don’t know. I think it’s not hard for you to make it, but with security concern.

>
>
> From: Alex Markuze [mailto:alex@weka.io<mailto:alex@weka.io>]
> Sent: Tuesday, October 21, 2014 11:16 PM
> To: Liang, Cunming
> Cc: dev@dpdk.org<mailto:dev@dpdk.org>
> Subject: Re: [dpdk-dev] nic loopback
>
> How can I set/query this bit (LLE(PFVMTXSW[n]), intel 82599 ) on ESX, or any
> other friendlier environment like Linux?
>
> On Tue, Oct 21, 2014 at 4:18 AM, Liang, Cunming
> <cunming.liang@intel.com<mailto:cunming.liang@intel.com><mailto:cunming.liang@intel.com<mailto:cunming.liang@intel.com>>> wrote:
>
>
> > -----Original Message-----
> > From: dev
> [mailto:dev-bounces@dpdk.org<mailto:dev-bounces@dpdk.org><mailto:dev-bounces@dpdk.org<mailto:dev-bounces@dpdk.org>>]
> > On Behalf Of Alex Markuze
> > Sent: Tuesday, October 21, 2014 12:24 AM
> > To: dev@dpdk.org<mailto:dev@dpdk.org><mailto:dev@dpdk.org<mailto:dev@dpdk.org>>
> > Subject: [dpdk-dev] nic loopback
> >
> > Hi,
> > I'm trying to send packets from an application to it self, meaning
> > smac  == dmac.
> > I'm working with intel 82599 virtual function. But it seems that these
> > packets are lost.
> >
> > Is there a software/hw limitation I'm missing here (some additional
> > anti-spoofing)? AFAIK modern NICs with sriov are mini switches so the
> > hw loopback should work, at least thats the theory.
> >
> [Liang, Cunming] You could have a check on register LLE(PFVMTXSW[n]).
> Which allow an individual pool to be able to send traffic and have it loopback
> to itself.
> >
> > Thanks.

Re: [dpdk-dev] nic loopback

[Thomas Monjalon]

2014-10-20 19:24, Alex Markuze:
> I'm trying to send packets from an application to it self, meaning smac  ==
> dmac.
> I'm working with intel 82599 virtual function. But it seems that these
> packets are lost.
> 
> Is there a software/hw limitation I'm missing here (some additional
> anti-spoofing)? AFAIK modern NICs with sriov are mini switches so the hw
> loopback should work, at least thats the theory.

I think you should look at these commits:

ixgbe: add Tx->Rx loopback mode for 82599
	http://dpdk.org/browse/dpdk/commit/?id=db035925617
app/testpmd: add loopback topology
	http://dpdk.org/browse/dpdk/commit/?id=3e2006d6186

-- 
Thomas

Re: [dpdk-dev] nic loopback

[Alex Markuze]

Thanks Thomas,
unfortunately these patches are only valid a pf*. This is also evident from
the ixgbe pmd code which is the only one looking at this bit (lpbk_mode).
The ixgbevf functions are agnostic to this capability.

*
http://www.intel.com/content/dam/doc/design-guide/82599-sr-iov-driver-companion-guide.pdf

On Tue, Oct 21, 2014 at 6:32 PM, Thomas Monjalon <thomas.monjalon@6wind.com>
wrote:

> 2014-10-20 19:24, Alex Markuze:
> > I'm trying to send packets from an application to it self, meaning smac
> ==
> > dmac.
> > I'm working with intel 82599 virtual function. But it seems that these
> > packets are lost.
> >
> > Is there a software/hw limitation I'm missing here (some additional
> > anti-spoofing)? AFAIK modern NICs with sriov are mini switches so the hw
> > loopback should work, at least thats the theory.
>
> I think you should look at these commits:
>
> ixgbe: add Tx->Rx loopback mode for 82599
>         http://dpdk.org/browse/dpdk/commit/?id=db035925617
> app/testpmd: add loopback topology
>         http://dpdk.org/browse/dpdk/commit/?id=3e2006d6186
>
> --
> Thomas
>

Re: [dpdk-dev] nic loopback

[Thomas Monjalon]

21/10/2014 18:54, Alex Markuze :,
> unfortunately these patches are only valid a pf*. This is also evident from
> the ixgbe pmd code which is the only one looking at this bit (lpbk_mode).
> The ixgbevf functions are agnostic to this capability.

And did you try testpmd with PORT_TOPOLOGY_LOOP (2nd link)?
It could help to understand what's going wrong.

> > ixgbe: add Tx->Rx loopback mode for 82599
> > 
> >         http://dpdk.org/browse/dpdk/commit/?id=db035925617
> > 
> > app/testpmd: add loopback topology
> > 
> >         http://dpdk.org/browse/dpdk/commit/?id=3e2006d6186

Please Alex, do not top post.
-- 
Thomas