[dpdk-users] i40eVF pmd vlan id handling

[dpdk-users] i40eVF pmd vlan id handling

[Dey, Souvik]

Hi All,
        I am using DPDK 18.11.2 and i40e PF linux driver on the host 2.4.6. I see there, when I enable DEV_RX_OFFLOAD_VLAN_FILTER from the DPDK app, and then configure the specific vlan id using rte_eth_dev_vlan_filter(). As per DPDK code by default when we do dev_configure, we call i40evf_init_vlan() and if we don't enable rxmode.offloads with DEV_RX_OFFLOAD_VLAN_STRIP, then we should send VIRTCHNL_OP_DISABLE_VLAN_STRIPPING command to the PF from the guest.

With more debugging enabled, when DPDK requests VLAN filtering by sending VIRTCHNL_OP_ADD_VLAN to the PF, then we see that the VF stripping is enabled also on Linux. If we don't add VLAN ID and send VIRTCHNL_OP_ADD_VLAN message down to the PF , then we do receive packets with VLAN ID set.
The same works fine in VmWare drivers, where we do receive VALN id in the packets if STRIP is disabled.

In the linux case, when we receive frames with VLAN headers, the vlan id is stripped at the PF, and the TCI will record the missing VLAN details when handed up to the DPDK driver.

With i40e debug enabled, it's clear to see the difference being reported in i40e_rxd_to_vlan_tci:


Example using VLAN on i40evf SR-IOV (vlan fails):
  PMD: i40e_rxd_to_vlan_tci(): Mbuf vlan_tci: 8, vlan_tci_outer: 0
  Port 0 pkt-len=60 nb-segs=1
    ETH:  src=00:10:E0:8D:A7:52 dst=FF:FF:FF:FF:FF:FF type=0x0806
    ARP:  hrd=1 proto=0x0800 hln=6 pln=4 op=1 (ARP Request)
          sha=00:10:E0:8D:A7:52 sip=8.8.8.102
          tha=00:00:00:00:00:00 tip=8.8.8.3

As the application requested tagging not be stripped, and the hardware driver was not able to disable strip, in my opinion either DPDK or linux driver should have a bug in handing this. Also as we see VmWare i40e driver working it could be either a extra config required for linux driver or a bug in linux driver.
I also tried to add a call to rte_vlan_insert() to reinstate the VLAN header using the data from TCI in i40e_rxd_to_vlan_tci(), but it is having performance impacts as expected. Also as rte_vlan_insert() does not support QINQ, it will have other issues too.

Can you please clarify if this is working for i40e vlan filter test and if we are missing something in the DPDK, like sending VIRTCHNL_OP_DISABLE_VLAN_STRIPPING after every VIRTCHNL_OP_ADD_VLAN if DEV_RX_OFFLOAD_VLAN_STRIP is not set by the DPDk app.

--
Regards,
Souvik

Re: [dpdk-users] i40eVF pmd vlan id handling

[Dey, Souvik]

On debugging further it looks like that linux driver enables vlan_striping by default when VIRTCHNL_OP_ADD_VLAN is sent to the PF.

static int i40e_vc_add_vlan_msg(struct i40e_vf *vf, u8 *msg)
{
      .....
      i40e_vlan_stripping_enable(vsi);
      ....
}

Due to this when ever we enable vlan on i40e the van is stripped by default while sending it to the Guest. This could be handled by the below patch in DPDK :
In file i40e_ethdev_vf.c
static int
i40evf_add_vlan(struct rte_eth_dev *dev, uint16_t vlanid)
{
      struct i40e_vf *vf = I40EVF_DEV_PRIVATE_TO_VF(dev->data->dev_private);
      struct virtchnl_vlan_filter_list *vlan_list;
      uint8_t cmd_buffer[sizeof(struct virtchnl_vlan_filter_list) +
                                          sizeof(uint16_t)];
      int err;
      struct vf_cmd_info args;

      vlan_list = (struct virtchnl_vlan_filter_list *)cmd_buffer;
      vlan_list->vsi_id = vf->vsi_res->vsi_id;
      vlan_list->num_elements = 1;
      vlan_list->vlan_id[0] = vlanid;

      args.ops = VIRTCHNL_OP_ADD_VLAN;
      args.in_args = (u8 *)&cmd_buffer;
      args.in_args_size = sizeof(cmd_buffer);
      args.out_buffer = vf->aq_resp;
      args.out_size = I40E_AQ_BUF_SZ;
      err = i40evf_execute_vf_cmd(dev, &args);
      if (err)
            PMD_DRV_LOG(ERR, "fail to execute command OP_ADD_VLAN");
+     if (!(dev_conf->rxmode.offloads & DEV_RX_OFFLOAD_VLAN_STRIP)) {
+           i40evf_disable_vlan_strip(dev);
+     }
      return err;
}
Or we need to call . vlan_offload_set after doing . vlan_filter_set from the DPDK app ?

--
Regards,
Souvik

From: Dey, Souvik
Sent: Monday, April 13, 2020 12:04 PM
To: dev@dpdk.org; users@dpdk.org; beilei.xing@intel.com
Cc: ferruh.yigit@intel.com; qi.z.zhang@intel.com
Subject: i40eVF pmd vlan id handling

Hi All,
        I am using DPDK 18.11.2 and i40e PF linux driver on the host 2.4.6. I see there, when I enable DEV_RX_OFFLOAD_VLAN_FILTER from the DPDK app, and then configure the specific vlan id using rte_eth_dev_vlan_filter(). As per DPDK code by default when we do dev_configure, we call i40evf_init_vlan() and if we don't enable rxmode.offloads with DEV_RX_OFFLOAD_VLAN_STRIP, then we should send VIRTCHNL_OP_DISABLE_VLAN_STRIPPING command to the PF from the guest.

With more debugging enabled, when DPDK requests VLAN filtering by sending VIRTCHNL_OP_ADD_VLAN to the PF, then we see that the VF stripping is enabled also on Linux. If we don't add VLAN ID and send VIRTCHNL_OP_ADD_VLAN message down to the PF , then we do receive packets with VLAN ID set.

The same works fine in VmWare drivers, where we do receive VALN id in the packets if STRIP is disabled.

In the linux case, when we receive frames with VLAN headers, the vlan id is stripped at the PF, and the TCI will record the missing VLAN details when handed up to the DPDK driver.

With i40e debug enabled, it's clear to see the difference being reported in i40e_rxd_to_vlan_tci:


Example using VLAN on i40evf SR-IOV (vlan fails):
  PMD: i40e_rxd_to_vlan_tci(): Mbuf vlan_tci: 8, vlan_tci_outer: 0
  Port 0 pkt-len=60 nb-segs=1
    ETH:  src=00:10:E0:8D:A7:52 dst=FF:FF:FF:FF:FF:FF type=0x0806
    ARP:  hrd=1 proto=0x0800 hln=6 pln=4 op=1 (ARP Request)
          sha=00:10:E0:8D:A7:52 sip=8.8.8.102
          tha=00:00:00:00:00:00 tip=8.8.8.3

As the application requested tagging not be stripped, and the hardware driver was not able to disable strip, in my opinion either DPDK or linux driver should have a bug in handing this. Also as we see VmWare i40e driver working it could be either a extra config required for linux driver or a bug in linux driver.

I also tried to add a call to rte_vlan_insert() to reinstate the VLAN header using the data from TCI in i40e_rxd_to_vlan_tci(), but it is having performance impacts as expected. Also as rte_vlan_insert() does not support QINQ, it will have other issues too.

Can you please clarify if this is working for i40e vlan filter test and if we are missing something in the DPDK, like sending VIRTCHNL_OP_DISABLE_VLAN_STRIPPING after every VIRTCHNL_OP_ADD_VLAN if DEV_RX_OFFLOAD_VLAN_STRIP is not set by the DPDk app.

--
Regards,
Souvik

Re: [dpdk-users] i40eVF pmd vlan id handling

[Xing, Beilei]

Hi Souvik,

With kernel PF + DPDK VF, enable vlan filter and disable vlan strip on VF side, is this your requirement?
If yes, I think you can do . vlan_offload_set after doing . vlan_filter_set on VF side.

BR,
Beilei

From: Dey, Souvik <sodey@rbbn.com>
Sent: Tuesday, April 14, 2020 4:04 AM
To: dev@dpdk.org; users@dpdk.org; Xing, Beilei <beilei.xing@intel.com>
Cc: Yigit, Ferruh <ferruh.yigit@intel.com>; Zhang, Qi Z <qi.z.zhang@intel.com>
Subject: RE: i40eVF pmd vlan id handling

On debugging further it looks like that linux driver enables vlan_striping by default when VIRTCHNL_OP_ADD_VLAN is sent to the PF.

static int i40e_vc_add_vlan_msg(struct i40e_vf *vf, u8 *msg)
{
      …..
      i40e_vlan_stripping_enable(vsi);
      ….
}

Due to this when ever we enable vlan on i40e the van is stripped by default while sending it to the Guest. This could be handled by the below patch in DPDK :
In file i40e_ethdev_vf.c
static int
i40evf_add_vlan(struct rte_eth_dev *dev, uint16_t vlanid)
{
      struct i40e_vf *vf = I40EVF_DEV_PRIVATE_TO_VF(dev->data->dev_private);
      struct virtchnl_vlan_filter_list *vlan_list;
      uint8_t cmd_buffer[sizeof(struct virtchnl_vlan_filter_list) +
                                          sizeof(uint16_t)];
      int err;
      struct vf_cmd_info args;

      vlan_list = (struct virtchnl_vlan_filter_list *)cmd_buffer;
      vlan_list->vsi_id = vf->vsi_res->vsi_id;
      vlan_list->num_elements = 1;
      vlan_list->vlan_id[0] = vlanid;

      args.ops = VIRTCHNL_OP_ADD_VLAN;
      args.in_args = (u8 *)&cmd_buffer;
      args.in_args_size = sizeof(cmd_buffer);
      args.out_buffer = vf->aq_resp;
      args.out_size = I40E_AQ_BUF_SZ;
      err = i40evf_execute_vf_cmd(dev, &args);
      if (err)
            PMD_DRV_LOG(ERR, "fail to execute command OP_ADD_VLAN");
+     if (!(dev_conf->rxmode.offloads & DEV_RX_OFFLOAD_VLAN_STRIP)) {
+           i40evf_disable_vlan_strip(dev);
+     }
      return err;
}
Or we need to call . vlan_offload_set after doing . vlan_filter_set from the DPDK app ?

--
Regards,
Souvik

From: Dey, Souvik
Sent: Monday, April 13, 2020 12:04 PM
To: dev@dpdk.org<mailto:dev@dpdk.org>; users@dpdk.org<mailto:users@dpdk.org>; beilei.xing@intel.com<mailto:beilei.xing@intel.com>
Cc: ferruh.yigit@intel.com<mailto:ferruh.yigit@intel.com>; qi.z.zhang@intel.com<mailto:qi.z.zhang@intel.com>
Subject: i40eVF pmd vlan id handling

Hi All,
        I am using DPDK 18.11.2 and i40e PF linux driver on the host 2.4.6. I see there, when I enable DEV_RX_OFFLOAD_VLAN_FILTER from the DPDK app, and then configure the specific vlan id using rte_eth_dev_vlan_filter(). As per DPDK code by default when we do dev_configure, we call i40evf_init_vlan() and if we don’t enable rxmode.offloads with DEV_RX_OFFLOAD_VLAN_STRIP, then we should send VIRTCHNL_OP_DISABLE_VLAN_STRIPPING command to the PF from the guest.

With more debugging enabled, when DPDK requests VLAN filtering by sending VIRTCHNL_OP_ADD_VLAN to the PF, then we see that the VF stripping is enabled also on Linux. If we don’t add VLAN ID and send VIRTCHNL_OP_ADD_VLAN message down to the PF , then we do receive packets with VLAN ID set.

The same works fine in VmWare drivers, where we do receive VALN id in the packets if STRIP is disabled.

In the linux case, when we receive frames with VLAN headers, the vlan id is stripped at the PF, and the TCI will record the missing VLAN details when handed up to the DPDK driver.

With i40e debug enabled, it's clear to see the difference being reported in i40e_rxd_to_vlan_tci:


Example using VLAN on i40evf SR-IOV (vlan fails):
  PMD: i40e_rxd_to_vlan_tci(): Mbuf vlan_tci: 8, vlan_tci_outer: 0
  Port 0 pkt-len=60 nb-segs=1
    ETH:  src=00:10:E0:8D:A7:52 dst=FF:FF:FF:FF:FF:FF type=0x0806
    ARP:  hrd=1 proto=0x0800 hln=6 pln=4 op=1 (ARP Request)
          sha=00:10:E0:8D:A7:52 sip=8.8.8.102
          tha=00:00:00:00:00:00 tip=8.8.8.3

As the application requested tagging not be stripped, and the hardware driver was not able to disable strip, in my opinion either DPDK or linux driver should have a bug in handing this. Also as we see VmWare i40e driver working it could be either a extra config required for linux driver or a bug in linux driver.

I also tried to add a call to rte_vlan_insert() to reinstate the VLAN header using the data from TCI in i40e_rxd_to_vlan_tci(), but it is having performance impacts as expected. Also as rte_vlan_insert() does not support QINQ, it will have other issues too.

Can you please clarify if this is working for i40e vlan filter test and if we are missing something in the DPDK, like sending VIRTCHNL_OP_DISABLE_VLAN_STRIPPING after every VIRTCHNL_OP_ADD_VLAN if DEV_RX_OFFLOAD_VLAN_STRIP is not set by the DPDk app.

--
Regards,
Souvik


________________________________
Notice: This e-mail together with any attachments may contain information of Ribbon Communications Inc. that is confidential and/or proprietary for the sole use of the intended recipient. Any review, disclosure, reliance or distribution by others or forwarding without express permission is strictly prohibited. If you are not the intended recipient, please notify the sender immediately and then delete all copies, including any attachments.
________________________________

Re: [dpdk-users] i40eVF pmd vlan id handling

[Dey, Souvik]

Hi Beilei,
      Yes , as per our requirement, we need to have vlan stripped disabled, when we enable vlan filter and add vlan to the PF from the guest, so that the vlan id comes upto the app in the packet itself without striping. We also tried to call rte_vlan_insert in the packet receive side, but that has huge performance impacts.

What we do from the DPDK app, is during the port configuration, we enable vlan_filter and disable vlan_strip as a part of dev_conf.rxmode.offloads. Then, when we have vlans from the app to be added to the PF, we call add_vlan with the vlan id to be added.
As a part of the add_vlan, the kernel PF driver enables vlan_strip too by default. Now in this case we cannot use the vlan_offload_set from the DPDK to disable the vlan_strip.
As per the code in rte_eth_dev_set_vlan_offload() , in this case if we pass the mask as ETH_VLAN_FILTER_OFFLOAD, and both cur and org will evaluate to be equal(as DEV_RX_OFFLOAD_VLAN_STRIP is already disabled during port conf) and VLAN_STRIP will not be disabled.

              cur = !!(offload_mask & ETH_VLAN_STRIP_OFFLOAD);
      org = !!(dev_offloads & DEV_RX_OFFLOAD_VLAN_STRIP);
      if (cur != org) {
           if (cur)
                 dev_offloads |= DEV_RX_OFFLOAD_VLAN_STRIP;
           else
                 dev_offloads &= ~DEV_RX_OFFLOAD_VLAN_STRIP;
           mask |= ETH_VLAN_STRIP_MASK;
      }

The only possibility will be that we keep VLAN_STRIP enabled during port configuration and then call set_vlan_offload to disable the VLAN_STRIP, by passing the mask as ETH_VLAN_FILTER_OFFLOAD .
But this will also have issues, when we try to add the 2nd vlan_id. As the PF support till 8/16(depends on PF driver version) vlan ids from the guest(without trust mode on) , the 2nd add_vlan call will again enable the vlan_strip in the PF.
This will happen as after the 1st call to the set_vlan_offload, on successful vlan_offload_set() , we will update the dev_conf.rxmode.offloads with the computed dev_offloads (dev_offloads &= ~DEV_RX_OFFLOAD_VLAN_STRIP).
dev->data->dev_conf.rxmode.offloads = dev_offloads;

So, we will face the same issue as descried above where cur and org will evaluate to be equal(as VLAN_STRIP already disabled in 1st  call) and VLAN_STRIP will not be disabled again.

Instead of that, if we call i40evf_disable_vlan_strip() when dev_conf.rxmode.offloads has VLAN_STRIP set to disabled after successful call of add_vlan from the pmd itself that will help in every scenario.
This scenario works fine if we use DPDK at the PF side too as we don’t enable VLAN_STRIP in add_vlan in the DPDK i40e PF.

--
Regards,
Souvik

From: Xing, Beilei <beilei.xing@intel.com>
Sent: Tuesday, April 14, 2020 2:37 AM
To: Dey, Souvik <sodey@rbbn.com>; dev@dpdk.org; users@dpdk.org
Cc: Yigit, Ferruh <ferruh.yigit@intel.com>; Zhang, Qi Z <qi.z.zhang@intel.com>
Subject: RE: i40eVF pmd vlan id handling

________________________________
NOTICE: This email was received from an EXTERNAL sender
________________________________

Hi Souvik,

With kernel PF + DPDK VF, enable vlan filter and disable vlan strip on VF side, is this your requirement?
If yes, I think you can do . vlan_offload_set after doing . vlan_filter_set on VF side.

BR,
Beilei

From: Dey, Souvik <sodey@rbbn.com<mailto:sodey@rbbn.com>>
Sent: Tuesday, April 14, 2020 4:04 AM
To: dev@dpdk.org<mailto:dev@dpdk.org>; users@dpdk.org<mailto:users@dpdk.org>; Xing, Beilei <beilei.xing@intel.com<mailto:beilei.xing@intel.com>>
Cc: Yigit, Ferruh <ferruh.yigit@intel.com<mailto:ferruh.yigit@intel.com>>; Zhang, Qi Z <qi.z.zhang@intel.com<mailto:qi.z.zhang@intel.com>>
Subject: RE: i40eVF pmd vlan id handling

On debugging further it looks like that linux driver enables vlan_striping by default when VIRTCHNL_OP_ADD_VLAN is sent to the PF.

static int i40e_vc_add_vlan_msg(struct i40e_vf *vf, u8 *msg)
{
      …..
      i40e_vlan_stripping_enable(vsi);
      ….
}

Due to this when ever we enable vlan on i40e the van is stripped by default while sending it to the Guest. This could be handled by the below patch in DPDK :
In file i40e_ethdev_vf.c
static int
i40evf_add_vlan(struct rte_eth_dev *dev, uint16_t vlanid)
{
      struct i40e_vf *vf = I40EVF_DEV_PRIVATE_TO_VF(dev->data->dev_private);
      struct virtchnl_vlan_filter_list *vlan_list;
      uint8_t cmd_buffer[sizeof(struct virtchnl_vlan_filter_list) +
                                          sizeof(uint16_t)];
      int err;
      struct vf_cmd_info args;

      vlan_list = (struct virtchnl_vlan_filter_list *)cmd_buffer;
      vlan_list->vsi_id = vf->vsi_res->vsi_id;
      vlan_list->num_elements = 1;
      vlan_list->vlan_id[0] = vlanid;

      args.ops = VIRTCHNL_OP_ADD_VLAN;
      args.in_args = (u8 *)&cmd_buffer;
      args.in_args_size = sizeof(cmd_buffer);
      args.out_buffer = vf->aq_resp;
      args.out_size = I40E_AQ_BUF_SZ;
      err = i40evf_execute_vf_cmd(dev, &args);
      if (err)
            PMD_DRV_LOG(ERR, "fail to execute command OP_ADD_VLAN");
+     if (!(dev_conf->rxmode.offloads & DEV_RX_OFFLOAD_VLAN_STRIP)) {
+           i40evf_disable_vlan_strip(dev);
+     }
      return err;
}
Or we need to call . vlan_offload_set after doing . vlan_filter_set from the DPDK app ?

--
Regards,
Souvik

From: Dey, Souvik
Sent: Monday, April 13, 2020 12:04 PM
To: dev@dpdk.org<mailto:dev@dpdk.org>; users@dpdk.org<mailto:users@dpdk.org>; beilei.xing@intel.com<mailto:beilei.xing@intel.com>
Cc: ferruh.yigit@intel.com<mailto:ferruh.yigit@intel.com>; qi.z.zhang@intel.com<mailto:qi.z.zhang@intel.com>
Subject: i40eVF pmd vlan id handling

Hi All,
        I am using DPDK 18.11.2 and i40e PF linux driver on the host 2.4.6. I see there, when I enable DEV_RX_OFFLOAD_VLAN_FILTER from the DPDK app, and then configure the specific vlan id using rte_eth_dev_vlan_filter(). As per DPDK code by default when we do dev_configure, we call i40evf_init_vlan() and if we don’t enable rxmode.offloads with DEV_RX_OFFLOAD_VLAN_STRIP, then we should send VIRTCHNL_OP_DISABLE_VLAN_STRIPPING command to the PF from the guest.

With more debugging enabled, when DPDK requests VLAN filtering by sending VIRTCHNL_OP_ADD_VLAN to the PF, then we see that the VF stripping is enabled also on Linux. If we don’t add VLAN ID and send VIRTCHNL_OP_ADD_VLAN message down to the PF , then we do receive packets with VLAN ID set.

The same works fine in VmWare drivers, where we do receive VALN id in the packets if STRIP is disabled.

In the linux case, when we receive frames with VLAN headers, the vlan id is stripped at the PF, and the TCI will record the missing VLAN details when handed up to the DPDK driver.

With i40e debug enabled, it's clear to see the difference being reported in i40e_rxd_to_vlan_tci:


Example using VLAN on i40evf SR-IOV (vlan fails):
  PMD: i40e_rxd_to_vlan_tci(): Mbuf vlan_tci: 8, vlan_tci_outer: 0
  Port 0 pkt-len=60 nb-segs=1
    ETH:  src=00:10:E0:8D:A7:52 dst=FF:FF:FF:FF:FF:FF type=0x0806
    ARP:  hrd=1 proto=0x0800 hln=6 pln=4 op=1 (ARP Request)
          sha=00:10:E0:8D:A7:52 sip=8.8.8.102<http://8.8.8.102>
          tha=00:00:00:00:00:00 tip=8.8.8.3<http://8.8.8.3>

As the application requested tagging not be stripped, and the hardware driver was not able to disable strip, in my opinion either DPDK or linux driver should have a bug in handing this. Also as we see VmWare i40e driver working it could be either a extra config required for linux driver or a bug in linux driver.

I also tried to add a call to rte_vlan_insert() to reinstate the VLAN header using the data from TCI in i40e_rxd_to_vlan_tci(), but it is having performance impacts as expected. Also as rte_vlan_insert() does not support QINQ, it will have other issues too.

Can you please clarify if this is working for i40e vlan filter test and if we are missing something in the DPDK, like sending VIRTCHNL_OP_DISABLE_VLAN_STRIPPING after every VIRTCHNL_OP_ADD_VLAN if DEV_RX_OFFLOAD_VLAN_STRIP is not set by the DPDk app.

--
Regards,
Souvik


________________________________
Notice: This e-mail together with any attachments may contain information of Ribbon Communications Inc. that is confidential and/or proprietary for the sole use of the intended recipient. Any review, disclosure, reliance or distribution by others or forwarding without express permission is strictly prohibited. If you are not the intended recipient, please notify the sender immediately and then delete all copies, including any attachments.
________________________________