Re: [dpdk-dev] ixgbe : query regarding your code changes for VF mac add

Re: [dpdk-dev] ixgbe : query regarding your code changes for VF mac add

[Ivan Boule]

Hi Santosh,

I do not get exactly what you attempt to do on a VF.
Are you first deleting the so-called permanent MAC address by a call to the
function ixgbevf_remove_mac_addr() ? This operation is not allowed.
Can you explain exactly the sequence of operations that are done, so that I
can understand how the test (memcmp(hw->mac.perm_addr, mac_addr,
sizeof(struct ether_addr)) == 0) in the function ixgbevf_add_mac_addr()
prevents them to be successfully performed.

Ivan

PS : please, can you CC your emails to dev@dpdk.org


2016-04-19 17:01 GMT+02:00 santosh <santosh.iitg@gmail.com>:

> Hi Ivan,
>
> Your following code changes causing issue in Vmware environment.
>
> ----------------------------------- -------------------
> ------------------------------
> + /*
> + * On a 82599 VF, adding again the same MAC addr is not an idempotent
> + * operation. Trap this case to avoid exhausting the [very limited]
> + * set of PF resources used to store VF MAC addresses.
> + */
> + if (memcmp(hw->mac.perm_addr, mac_addr, sizeof(struct ether_addr)) == 0)
> + return;
>   diag = ixgbevf_set_uc_addr_vf(hw, 2, mac_addr->addr_bytes);
> --------------------------------------------------------------------
> ------------- -------
>
>
> Issue:
> At CLI , we have provision to add /del MAC of an interface.
> During MAC delete, existing MAC is deleted and default MAC is applied.
> This default MAC is not being applied in VMware environment
> successfully due to "return" statement
> in your above code changes. As a result traffic is stopped completely.
> If I remove above
> "return" statement then traffic continues to flow after MAC delete.
>
> Please let me know your suggestion to handle this scenario .  If I
> remove "return" what will be the consequences ?
>
> If removing "return" statement is not good idea then what are other
> way to handle MAC delete scenario ?  we have only 1 VF per PF in our
> setup as of now.
>
>
> Thanks
> Santosh
>



-- 
Ivan BOULE
6WIND
Software Engineer

Tel: +33 1 39 30 92 47
Mob: + 33 6 77 25 26 38
Fax: +33 1 39 30 92 11
ivan.boule@6wind.com
www.6wind.com
Join the Multicore Packet Processing Forum:
www.multicorepacketprocessing.com

Ce courriel ainsi que toutes les pièces jointes, est uniquement destiné à
son ou ses destinataires. Il contient des informations confidentielles qui
sont la propriété de 6WIND. Toute révélation, distribution ou copie des
informations qu'il contient est strictement interdite. Si vous avez reçu ce
message par erreur, veuillez immédiatement le signaler à l'émetteur et
détruire toutes les données reçues.

This e-mail message, including any attachments, is for the sole use of the
intended recipient(s) and contains information that is confidential and
proprietary to 6WIND. All unauthorized review, use, disclosure or
distribution is prohibited. If you are not the intended recipient, please
contact the sender by reply e-mail and destroy all copies of the original
message.

Re: [dpdk-dev] ixgbe : query regarding your code changes for VF mac add

[santosh]

Hi Ivan,

Thanks for your response.

Let me explain you the issue that we are facing on our virtual router
in VMware environment.

1. We are using ixgbe driver , SRIOV enabled .
    root@localhost:~# lspci
      .... "Intel Corporation 82599 Ethernet Controller Virtual Function"

2.  "mx86-bgl-1-r1"  is our router under testing and  R2 is a standard router.

      mx86-bgl-1-r1 is connected to R2
      mx86-bgl-1-r1 (10.1.1.103)------------------>R2(10.1.1.101)

2. This time ping test passes

root@mx86-bgl-1-r1# show interfaces
ge-0/0/0 {
    unit 0 {
        family inet {
            address 10.1.1.103/24;
        }
    }
}

[edit]
root@mx86-bgl-1-r1#
root@mx86-bgl-1-r1# run ping 10.1.1.101
PING 10.1.1.101 (10.1.1.101): 56 data bytes
64 bytes from 10.1.1.101: icmp_seq=0 ttl=64 time=0.980 ms
64 bytes from 10.1.1.101: icmp_seq=1 ttl=64 time=1.433 ms


3.  Default MAC address of interface ge-0/0/0  is as shown below:

root@mx86-bgl-1-r1# run show interfaces ge-0/0/0 extensive | grep current
  Current address: 02:06:0a:0e:ff:f0, Hardware address: 02:06:0a:0e:ff:f0

[edit]
root@mx86-bgl-1-r1#


4.  Now I am changing MAC. Ping works this time also

root@mx86-bgl-1-r1# set interfaces ge-0/0/0 mac 02:06:0a:0a:ff:f0

[edit]
root@mx86-bgl-1-r1# commit
commit complete

[edit]
root@mx86-bgl-1-r1# run show interfaces ge-0/0/0 extensive | grep current
  Current address: 02:06:0a:0a:ff:f0, Hardware address: 02:06:0a:0e:ff:f0

[edit]
root@mx86-bgl-1-r1# show interfaces
ge-0/0/0 {
    mac 02:06:0a:0a:ff:f0;
    unit 0 {
        family inet {
            address 10.1.1.103/24;
        }
    }
}

[edit]
root@mx86-bgl-1-r1#
root@mx86-bgl-1-r1# run ping 10.1.1.101
PING 10.1.1.101 (10.1.1.101): 56 data bytes
64 bytes from 10.1.1.101: icmp_seq=0 ttl=64 time=1.338 ms
64 bytes from 10.1.1.101: icmp_seq=1 ttl=64 time=8.832 ms
64 bytes from 10.1.1.101: icmp_seq=2 ttl=64 time=1.292 ms


5.  I am deleting the above configured MAC.
     In this case newly configured MAC as shown above will be deleted
and Default MAC (02:06:0a:0e:ff:f0)
      will be applied.  Ping fails at this step.  "return statement
added by you is not allowing to configure default MAC.


[edit]
root@mx86-bgl-1-r1# delete interfaces ge-0/0/0 mac

[edit]
root@mx86-bgl-1-r1# commit
commit complete

[edit]
root@mx86-bgl-1-r1# show interfaces
ge-0/0/0 {
    unit 0 {
        family inet {
            address 10.1.1.103/24;
        }
    }
}

[edit]

root@mx86-bgl-1-r1# run show interfaces ge-0/0/0 extensive | grep
current     // Displays value from router's local database not from
NIC.
  Current address: 02:06:0a:0e:ff:f0, Hardware address: 02:06:0a:0e:ff:f0

[edit]
root@mx86-bgl-1-r1# run ping 10.1.1.101
PING 10.1.1.101 (10.1.1.101): 56 data bytes
^C
2 packets transmitted, 0 packets received, 100% packet loss

[edit]
root@mx86-bgl-1-r1#


7. LOGs:    (I have added a debug log just before the return statement
that you place)

Log o/p in failure case
....
Santosh ixgbevf_add_mac_addr returning
....

code changes:
-----------------------------
ixgbevf_add_mac_addr(....)  {
...
    if (memcmp(hw->mac.perm_addr, mac_addr, sizeof(struct ether_addr)) == 0) {
        PMD_DRV_LOG(DEBUG, "Existing MAC \n");
        printf("Santosh %s returning \n",__FUNCTION__);
        return;
    }


8.  If I remove the above "return" statement, build the driver, loaded
in router and repeat the steps-2 to steps-5 of MAC add and MAC delete
on our router then ping passes.

root@mx86-bgl-1-r1# run ping 10.1.1.101
PING 10.1.1.101 (10.1.1.101): 56 data bytes
64 bytes from 10.1.1.101: icmp_seq=0 ttl=64 time=2.356 ms
64 bytes from 10.1.1.101: icmp_seq=1 ttl=64 time=1.415 ms
64 bytes from 10.1.1.101: icmp_seq=2 ttl=64 time=1.226 ms
^C
--- 10.1.1.101 ping statistics ---
3 packets transmitted, 3 packets received, 0% packet loss
round-trip min/avg/max/stddev = 1.226/1.666/2.356/0.494 ms


9.  Please let me know whether is it  fine to remove that return
statement added by you in  "ixgbevf_add_mac_addr"  ?



Thanks & Regards,
Santosh

On Wed, Apr 20, 2016 at 3:31 AM, Ivan Boule <ivan.boule@6wind.com> wrote:
> Hi Santosh,
>
> I do not get exactly what you attempt to do on a VF.
> Are you first deleting the so-called permanent MAC address by a call to the
> function ixgbevf_remove_mac_addr() ? This operation is not allowed.
> Can you explain exactly the sequence of operations that are done, so that I
> can understand how the test (memcmp(hw->mac.perm_addr, mac_addr,
> sizeof(struct ether_addr)) == 0) in the function ixgbevf_add_mac_addr()
> prevents them to be successfully performed.
>
> Ivan
>
> PS : please, can you CC your emails to dev@dpdk.org
>
>
> 2016-04-19 17:01 GMT+02:00 santosh <santosh.iitg@gmail.com>:
>>
>> Hi Ivan,
>>
>> Your following code changes causing issue in Vmware environment.
>>
>> ----------------------------------- -------------------
>> ------------------------------
>> + /*
>> + * On a 82599 VF, adding again the same MAC addr is not an idempotent
>> + * operation. Trap this case to avoid exhausting the [very limited]
>> + * set of PF resources used to store VF MAC addresses.
>> + */
>> + if (memcmp(hw->mac.perm_addr, mac_addr, sizeof(struct ether_addr)) == 0)
>> + return;
>>   diag = ixgbevf_set_uc_addr_vf(hw, 2, mac_addr->addr_bytes);
>> --------------------------------------------------------------------
>> ------------- -------
>>
>>
>> Issue:
>> At CLI , we have provision to add /del MAC of an interface.
>> During MAC delete, existing MAC is deleted and default MAC is applied.
>> This default MAC is not being applied in VMware environment
>> successfully due to "return" statement
>> in your above code changes. As a result traffic is stopped completely.
>> If I remove above
>> "return" statement then traffic continues to flow after MAC delete.
>>
>> Please let me know your suggestion to handle this scenario .  If I
>> remove "return" what will be the consequences ?
>>
>> If removing "return" statement is not good idea then what are other
>> way to handle MAC delete scenario ?  we have only 1 VF per PF in our
>> setup as of now.
>>
>>
>> Thanks
>> Santosh
>
>
>
>
> --
> Ivan BOULE
> 6WIND
> Software Engineer
>
> Tel: +33 1 39 30 92 47
> Mob: + 33 6 77 25 26 38
> Fax: +33 1 39 30 92 11
> ivan.boule@6wind.com
> www.6wind.com
> Join the Multicore Packet Processing Forum:
> www.multicorepacketprocessing.com
>
> Ce courriel ainsi que toutes les pièces jointes, est uniquement destiné à
> son ou ses destinataires. Il contient des informations confidentielles qui
> sont la propriété de 6WIND. Toute révélation, distribution ou copie des
> informations qu'il contient est strictement interdite. Si vous avez reçu ce
> message par erreur, veuillez immédiatement le signaler à l'émetteur et
> détruire toutes les données reçues.
>
> This e-mail message, including any attachments, is for the sole use of the
> intended recipient(s) and contains information that is confidential and
> proprietary to 6WIND. All unauthorized review, use, disclosure or
> distribution is prohibited. If you are not the intended recipient, please
> contact the sender by reply e-mail and destroy all copies of the original
> message.
>

Re: [dpdk-dev] ixgbe : query regarding your code changes for VF mac add

[santosh]

Hi Ivan and team,

Please respond to my last mail and  let me know if there is any
alternate way to handle this.
Our release is in pending due to this issue.


Thanks & Regards
Santosh

On Wed, Apr 20, 2016 at 2:35 PM, santosh <santosh.iitg@gmail.com> wrote:
> Hi Ivan,
>
> Thanks for your response.
>
> Let me explain you the issue that we are facing on our virtual router
> in VMware environment.
>
> 1. We are using ixgbe driver , SRIOV enabled .
>     root@localhost:~# lspci
>       .... "Intel Corporation 82599 Ethernet Controller Virtual Function"
>
> 2.  "mx86-bgl-1-r1"  is our router under testing and  R2 is a standard router.
>
>       mx86-bgl-1-r1 is connected to R2
>       mx86-bgl-1-r1 (10.1.1.103)------------------>R2(10.1.1.101)
>
> 2. This time ping test passes
>
> root@mx86-bgl-1-r1# show interfaces
> ge-0/0/0 {
>     unit 0 {
>         family inet {
>             address 10.1.1.103/24;
>         }
>     }
> }
>
> [edit]
> root@mx86-bgl-1-r1#
> root@mx86-bgl-1-r1# run ping 10.1.1.101
> PING 10.1.1.101 (10.1.1.101): 56 data bytes
> 64 bytes from 10.1.1.101: icmp_seq=0 ttl=64 time=0.980 ms
> 64 bytes from 10.1.1.101: icmp_seq=1 ttl=64 time=1.433 ms
>
>
> 3.  Default MAC address of interface ge-0/0/0  is as shown below:
>
> root@mx86-bgl-1-r1# run show interfaces ge-0/0/0 extensive | grep current
>   Current address: 02:06:0a:0e:ff:f0, Hardware address: 02:06:0a:0e:ff:f0
>
> [edit]
> root@mx86-bgl-1-r1#
>
>
> 4.  Now I am changing MAC. Ping works this time also
>
> root@mx86-bgl-1-r1# set interfaces ge-0/0/0 mac 02:06:0a:0a:ff:f0
>
> [edit]
> root@mx86-bgl-1-r1# commit
> commit complete
>
> [edit]
> root@mx86-bgl-1-r1# run show interfaces ge-0/0/0 extensive | grep current
>   Current address: 02:06:0a:0a:ff:f0, Hardware address: 02:06:0a:0e:ff:f0
>
> [edit]
> root@mx86-bgl-1-r1# show interfaces
> ge-0/0/0 {
>     mac 02:06:0a:0a:ff:f0;
>     unit 0 {
>         family inet {
>             address 10.1.1.103/24;
>         }
>     }
> }
>
> [edit]
> root@mx86-bgl-1-r1#
> root@mx86-bgl-1-r1# run ping 10.1.1.101
> PING 10.1.1.101 (10.1.1.101): 56 data bytes
> 64 bytes from 10.1.1.101: icmp_seq=0 ttl=64 time=1.338 ms
> 64 bytes from 10.1.1.101: icmp_seq=1 ttl=64 time=8.832 ms
> 64 bytes from 10.1.1.101: icmp_seq=2 ttl=64 time=1.292 ms
>
>
> 5.  I am deleting the above configured MAC.
>      In this case newly configured MAC as shown above will be deleted
> and Default MAC (02:06:0a:0e:ff:f0)
>       will be applied.  Ping fails at this step.  "return statement
> added by you is not allowing to configure default MAC.
>
>
> [edit]
> root@mx86-bgl-1-r1# delete interfaces ge-0/0/0 mac
>
> [edit]
> root@mx86-bgl-1-r1# commit
> commit complete
>
> [edit]
> root@mx86-bgl-1-r1# show interfaces
> ge-0/0/0 {
>     unit 0 {
>         family inet {
>             address 10.1.1.103/24;
>         }
>     }
> }
>
> [edit]
>
> root@mx86-bgl-1-r1# run show interfaces ge-0/0/0 extensive | grep
> current     // Displays value from router's local database not from
> NIC.
>   Current address: 02:06:0a:0e:ff:f0, Hardware address: 02:06:0a:0e:ff:f0
>
> [edit]
> root@mx86-bgl-1-r1# run ping 10.1.1.101
> PING 10.1.1.101 (10.1.1.101): 56 data bytes
> ^C
> 2 packets transmitted, 0 packets received, 100% packet loss
>
> [edit]
> root@mx86-bgl-1-r1#
>
>
> 7. LOGs:    (I have added a debug log just before the return statement
> that you place)
>
> Log o/p in failure case
> ....
> Santosh ixgbevf_add_mac_addr returning
> ....
>
> code changes:
> -----------------------------
> ixgbevf_add_mac_addr(....)  {
> ...
>     if (memcmp(hw->mac.perm_addr, mac_addr, sizeof(struct ether_addr)) == 0) {
>         PMD_DRV_LOG(DEBUG, "Existing MAC \n");
>         printf("Santosh %s returning \n",__FUNCTION__);
>         return;
>     }
>
>
> 8.  If I remove the above "return" statement, build the driver, loaded
> in router and repeat the steps-2 to steps-5 of MAC add and MAC delete
> on our router then ping passes.
>
> root@mx86-bgl-1-r1# run ping 10.1.1.101
> PING 10.1.1.101 (10.1.1.101): 56 data bytes
> 64 bytes from 10.1.1.101: icmp_seq=0 ttl=64 time=2.356 ms
> 64 bytes from 10.1.1.101: icmp_seq=1 ttl=64 time=1.415 ms
> 64 bytes from 10.1.1.101: icmp_seq=2 ttl=64 time=1.226 ms
> ^C
> --- 10.1.1.101 ping statistics ---
> 3 packets transmitted, 3 packets received, 0% packet loss
> round-trip min/avg/max/stddev = 1.226/1.666/2.356/0.494 ms
>
>
> 9.  Please let me know whether is it  fine to remove that return
> statement added by you in  "ixgbevf_add_mac_addr"  ?
>
>
>
> Thanks & Regards,
> Santosh
>
> On Wed, Apr 20, 2016 at 3:31 AM, Ivan Boule <ivan.boule@6wind.com> wrote:
>> Hi Santosh,
>>
>> I do not get exactly what you attempt to do on a VF.
>> Are you first deleting the so-called permanent MAC address by a call to the
>> function ixgbevf_remove_mac_addr() ? This operation is not allowed.
>> Can you explain exactly the sequence of operations that are done, so that I
>> can understand how the test (memcmp(hw->mac.perm_addr, mac_addr,
>> sizeof(struct ether_addr)) == 0) in the function ixgbevf_add_mac_addr()
>> prevents them to be successfully performed.
>>
>> Ivan
>>
>> PS : please, can you CC your emails to dev@dpdk.org
>>
>>
>> 2016-04-19 17:01 GMT+02:00 santosh <santosh.iitg@gmail.com>:
>>>
>>> Hi Ivan,
>>>
>>> Your following code changes causing issue in Vmware environment.
>>>
>>> ----------------------------------- -------------------
>>> ------------------------------
>>> + /*
>>> + * On a 82599 VF, adding again the same MAC addr is not an idempotent
>>> + * operation. Trap this case to avoid exhausting the [very limited]
>>> + * set of PF resources used to store VF MAC addresses.
>>> + */
>>> + if (memcmp(hw->mac.perm_addr, mac_addr, sizeof(struct ether_addr)) == 0)
>>> + return;
>>>   diag = ixgbevf_set_uc_addr_vf(hw, 2, mac_addr->addr_bytes);
>>> --------------------------------------------------------------------
>>> ------------- -------
>>>
>>>
>>> Issue:
>>> At CLI , we have provision to add /del MAC of an interface.
>>> During MAC delete, existing MAC is deleted and default MAC is applied.
>>> This default MAC is not being applied in VMware environment
>>> successfully due to "return" statement
>>> in your above code changes. As a result traffic is stopped completely.
>>> If I remove above
>>> "return" statement then traffic continues to flow after MAC delete.
>>>
>>> Please let me know your suggestion to handle this scenario .  If I
>>> remove "return" what will be the consequences ?
>>>
>>> If removing "return" statement is not good idea then what are other
>>> way to handle MAC delete scenario ?  we have only 1 VF per PF in our
>>> setup as of now.
>>>
>>>
>>> Thanks
>>> Santosh
>>
>>
>>
>>
>> --
>> Ivan BOULE
>> 6WIND
>> Software Engineer
>>
>> Tel: +33 1 39 30 92 47
>> Mob: + 33 6 77 25 26 38
>> Fax: +33 1 39 30 92 11
>> ivan.boule@6wind.com
>> www.6wind.com
>> Join the Multicore Packet Processing Forum:
>> www.multicorepacketprocessing.com
>>
>> Ce courriel ainsi que toutes les pièces jointes, est uniquement destiné à
>> son ou ses destinataires. Il contient des informations confidentielles qui
>> sont la propriété de 6WIND. Toute révélation, distribution ou copie des
>> informations qu'il contient est strictement interdite. Si vous avez reçu ce
>> message par erreur, veuillez immédiatement le signaler à l'émetteur et
>> détruire toutes les données reçues.
>>
>> This e-mail message, including any attachments, is for the sole use of the
>> intended recipient(s) and contains information that is confidential and
>> proprietary to 6WIND. All unauthorized review, use, disclosure or
>> distribution is prohibited. If you are not the intended recipient, please
>> contact the sender by reply e-mail and destroy all copies of the original
>> message.
>>

Re: [dpdk-dev] ixgbe : query regarding your code changes for VF mac add

[Ivan Boule]

Hi Santosh,

To explain what's really going on in your use case, it would be better 
to add traces in the following functions of the ixgbevf PMD:

Ivan

At the end of eth_ixgbevf_dev_init()

   printf("%s portid=%d mac=%02X:%02X:%02X:%02X:%02X:%02X\n",
          __func__, dev->data->port_id,
          perm_addr->addr_bytes[0],
	 perm_addr->addr_bytes[1],
	 perm_addr->addr_bytes[2],
	 perm_addr->addr_bytes[3],
	 perm_addr->addr_bytes[4],
	 perm_addr->addr_bytes[5]);

At the begining of ixgbevf_add_mac_addr()

    printf("%s portid=%d mac=%02X:%02X:%02X:%02X:%02X:%02X\n",
           __func__, dev->data->port_id,
           mac_addr->addr_bytes[0],
	  mac_addr->addr_bytes[1],
	  mac_addr->addr_bytes[2],
	  mac_addr->addr_bytes[3],
	  mac_addr->addr_bytes[4],
	  mac_addr->addr_bytes[5]);

At the begining of ixgbevf_remove_mac_addr()

  printf("%s portid=%d index=%d mac=%02X:%02X:%02X:%02X:%02X:%02X\n",
         __func__, dev->data->port_id, index,
         dev->data->mac_addrs[i].addr_bytes[0],
	dev->data->mac_addrs[i].addr_bytes[1],
	dev->data->mac_addrs[i].addr_bytes[2],
	dev->data->mac_addrs[i].addr_bytes[3],
	dev->data->mac_addrs[i].addr_bytes[4],
	dev->data->mac_addrs[i].addr_bytes[5]);

At the begining of ixgbevf_set_default_mac_addr()

    printf("%s portid=%d mac=%02X:%02X:%02X:%02X:%02X:%02X\n",
           __func__, dev->data->port_id,
           addr->addr_bytes[0],
	  addr->addr_bytes[1],
	  addr->addr_bytes[2],
	  addr->addr_bytes[3],
	  addr->addr_bytes[4],
	  addr->addr_bytes[5]);

On 04/20/2016 11:05 AM, santosh wrote:
> Hi Ivan,
>
> Thanks for your response.
>
> Let me explain you the issue that we are facing on our virtual router
> in VMware environment.
>
> 1. We are using ixgbe driver , SRIOV enabled .
>      root@localhost:~# lspci
>        .... "Intel Corporation 82599 Ethernet Controller Virtual Function"
>
> 2.  "mx86-bgl-1-r1"  is our router under testing and  R2 is a standard router.
>
>        mx86-bgl-1-r1 is connected to R2
>        mx86-bgl-1-r1 (10.1.1.103)------------------>R2(10.1.1.101)
>
> 2. This time ping test passes
>
> root@mx86-bgl-1-r1# show interfaces
> ge-0/0/0 {
>      unit 0 {
>          family inet {
>              address 10.1.1.103/24;
>          }
>      }
> }
>
> [edit]
> root@mx86-bgl-1-r1#
> root@mx86-bgl-1-r1# run ping 10.1.1.101
> PING 10.1.1.101 (10.1.1.101): 56 data bytes
> 64 bytes from 10.1.1.101: icmp_seq=0 ttl=64 time=0.980 ms
> 64 bytes from 10.1.1.101: icmp_seq=1 ttl=64 time=1.433 ms
>
>
> 3.  Default MAC address of interface ge-0/0/0  is as shown below:
>
> root@mx86-bgl-1-r1# run show interfaces ge-0/0/0 extensive | grep current
>    Current address: 02:06:0a:0e:ff:f0, Hardware address: 02:06:0a:0e:ff:f0
>
> [edit]
> root@mx86-bgl-1-r1#
>
>
> 4.  Now I am changing MAC. Ping works this time also
>
> root@mx86-bgl-1-r1# set interfaces ge-0/0/0 mac 02:06:0a:0a:ff:f0
>
> [edit]
> root@mx86-bgl-1-r1# commit
> commit complete
>
> [edit]
> root@mx86-bgl-1-r1# run show interfaces ge-0/0/0 extensive | grep current
>    Current address: 02:06:0a:0a:ff:f0, Hardware address: 02:06:0a:0e:ff:f0
>
> [edit]
> root@mx86-bgl-1-r1# show interfaces
> ge-0/0/0 {
>      mac 02:06:0a:0a:ff:f0;
>      unit 0 {
>          family inet {
>              address 10.1.1.103/24;
>          }
>      }
> }
>
> [edit]
> root@mx86-bgl-1-r1#
> root@mx86-bgl-1-r1# run ping 10.1.1.101
> PING 10.1.1.101 (10.1.1.101): 56 data bytes
> 64 bytes from 10.1.1.101: icmp_seq=0 ttl=64 time=1.338 ms
> 64 bytes from 10.1.1.101: icmp_seq=1 ttl=64 time=8.832 ms
> 64 bytes from 10.1.1.101: icmp_seq=2 ttl=64 time=1.292 ms
>
>
> 5.  I am deleting the above configured MAC.
>       In this case newly configured MAC as shown above will be deleted
> and Default MAC (02:06:0a:0e:ff:f0)
>        will be applied.  Ping fails at this step.  "return statement
> added by you is not allowing to configure default MAC.
>
>
> [edit]
> root@mx86-bgl-1-r1# delete interfaces ge-0/0/0 mac
>
> [edit]
> root@mx86-bgl-1-r1# commit
> commit complete
>
> [edit]
> root@mx86-bgl-1-r1# show interfaces
> ge-0/0/0 {
>      unit 0 {
>          family inet {
>              address 10.1.1.103/24;
>          }
>      }
> }
>
> [edit]
>
> root@mx86-bgl-1-r1# run show interfaces ge-0/0/0 extensive | grep
> current     // Displays value from router's local database not from
> NIC.
>    Current address: 02:06:0a:0e:ff:f0, Hardware address: 02:06:0a:0e:ff:f0
>
> [edit]
> root@mx86-bgl-1-r1# run ping 10.1.1.101
> PING 10.1.1.101 (10.1.1.101): 56 data bytes
> ^C
> 2 packets transmitted, 0 packets received, 100% packet loss
>
> [edit]
> root@mx86-bgl-1-r1#
>
>
> 7. LOGs:    (I have added a debug log just before the return statement
> that you place)
>
> Log o/p in failure case
> ....
> Santosh ixgbevf_add_mac_addr returning
> ....
>
> code changes:
> -----------------------------
> ixgbevf_add_mac_addr(....)  {
> ...
>      if (memcmp(hw->mac.perm_addr, mac_addr, sizeof(struct ether_addr)) == 0) {
>          PMD_DRV_LOG(DEBUG, "Existing MAC \n");
>          printf("Santosh %s returning \n",__FUNCTION__);
>          return;
>      }
>
>
> 8.  If I remove the above "return" statement, build the driver, loaded
> in router and repeat the steps-2 to steps-5 of MAC add and MAC delete
> on our router then ping passes.
>
> root@mx86-bgl-1-r1# run ping 10.1.1.101
> PING 10.1.1.101 (10.1.1.101): 56 data bytes
> 64 bytes from 10.1.1.101: icmp_seq=0 ttl=64 time=2.356 ms
> 64 bytes from 10.1.1.101: icmp_seq=1 ttl=64 time=1.415 ms
> 64 bytes from 10.1.1.101: icmp_seq=2 ttl=64 time=1.226 ms
> ^C
> --- 10.1.1.101 ping statistics ---
> 3 packets transmitted, 3 packets received, 0% packet loss
> round-trip min/avg/max/stddev = 1.226/1.666/2.356/0.494 ms
>
>
> 9.  Please let me know whether is it  fine to remove that return
> statement added by you in  "ixgbevf_add_mac_addr"  ?
>
>
>
> Thanks & Regards,
> Santosh
>
> On Wed, Apr 20, 2016 at 3:31 AM, Ivan Boule <ivan.boule@6wind.com> wrote:
>> Hi Santosh,
>>
>> I do not get exactly what you attempt to do on a VF.
>> Are you first deleting the so-called permanent MAC address by a call to the
>> function ixgbevf_remove_mac_addr() ? This operation is not allowed.
>> Can you explain exactly the sequence of operations that are done, so that I
>> can understand how the test (memcmp(hw->mac.perm_addr, mac_addr,
>> sizeof(struct ether_addr)) == 0) in the function ixgbevf_add_mac_addr()
>> prevents them to be successfully performed.
>>
>> Ivan
>>
>> PS : please, can you CC your emails to dev@dpdk.org
>>
>>
>> 2016-04-19 17:01 GMT+02:00 santosh <santosh.iitg@gmail.com>:
>>>
>>> Hi Ivan,
>>>
>>> Your following code changes causing issue in Vmware environment.
>>>
>>> ----------------------------------- -------------------
>>> ------------------------------
>>> + /*
>>> + * On a 82599 VF, adding again the same MAC addr is not an idempotent
>>> + * operation. Trap this case to avoid exhausting the [very limited]
>>> + * set of PF resources used to store VF MAC addresses.
>>> + */
>>> + if (memcmp(hw->mac.perm_addr, mac_addr, sizeof(struct ether_addr)) == 0)
>>> + return;
>>>    diag = ixgbevf_set_uc_addr_vf(hw, 2, mac_addr->addr_bytes);
>>> --------------------------------------------------------------------
>>> ------------- -------
>>>
>>>
>>> Issue:
>>> At CLI , we have provision to add /del MAC of an interface.
>>> During MAC delete, existing MAC is deleted and default MAC is applied.
>>> This default MAC is not being applied in VMware environment
>>> successfully due to "return" statement
>>> in your above code changes. As a result traffic is stopped completely.
>>> If I remove above
>>> "return" statement then traffic continues to flow after MAC delete.
>>>
>>> Please let me know your suggestion to handle this scenario .  If I
>>> remove "return" what will be the consequences ?
>>>
>>> If removing "return" statement is not good idea then what are other
>>> way to handle MAC delete scenario ?  we have only 1 VF per PF in our
>>> setup as of now.
>>>
>>>
>>> Thanks
>>> Santosh
>>
>>
>>
>>
>> --
>> Ivan BOULE
>> 6WIND
>> Software Engineer
>>
>> Tel: +33 1 39 30 92 47
>> Mob: + 33 6 77 25 26 38
>> Fax: +33 1 39 30 92 11
>> ivan.boule@6wind.com
>> www.6wind.com
>> Join the Multicore Packet Processing Forum:
>> www.multicorepacketprocessing.com
>>
>> Ce courriel ainsi que toutes les pièces jointes, est uniquement destiné à
>> son ou ses destinataires. Il contient des informations confidentielles qui
>> sont la propriété de 6WIND. Toute révélation, distribution ou copie des
>> informations qu'il contient est strictement interdite. Si vous avez reçu ce
>> message par erreur, veuillez immédiatement le signaler à l'émetteur et
>> détruire toutes les données reçues.
>>
>> This e-mail message, including any attachments, is for the sole use of the
>> intended recipient(s) and contains information that is confidential and
>> proprietary to 6WIND. All unauthorized review, use, disclosure or
>> distribution is prohibited. If you are not the intended recipient, please
>> contact the sender by reply e-mail and destroy all copies of the original
>> message.
>>


-- 
Ivan Boule
6WIND Development Engineer

Re: [dpdk-dev] ixgbe : query regarding your code changes for VF mac add

[Ivan Boule]

Hi Santosh,

My job at 6WIND does not consist in answering to DPDK questions. In 
general, I have other priorities, including vacations...
In the meantime, nobody prevents you to add traces in the code to really 
understand what happens, as suggested in my last answer.

Regards,
Ivan

On 04/21/2016 07:55 AM, santosh wrote:
> Hi Ivan and team,
>
> Please respond to my last mail and  let me know if there is any
> alternate way to handle this.
> Our release is in pending due to this issue.
>
>
> Thanks & Regards
> Santosh
>
> On Wed, Apr 20, 2016 at 2:35 PM, santosh <santosh.iitg@gmail.com> wrote:
>> Hi Ivan,
>>
>> Thanks for your response.
>>
>> Let me explain you the issue that we are facing on our virtual router
>> in VMware environment.
>>
>> 1. We are using ixgbe driver , SRIOV enabled .
>>      root@localhost:~# lspci
>>        .... "Intel Corporation 82599 Ethernet Controller Virtual Function"
>>
>> 2.  "mx86-bgl-1-r1"  is our router under testing and  R2 is a standard router.
>>
>>        mx86-bgl-1-r1 is connected to R2
>>        mx86-bgl-1-r1 (10.1.1.103)------------------>R2(10.1.1.101)
>>
>> 2. This time ping test passes
>>
>> root@mx86-bgl-1-r1# show interfaces
>> ge-0/0/0 {
>>      unit 0 {
>>          family inet {
>>              address 10.1.1.103/24;
>>          }
>>      }
>> }
>>
>> [edit]
>> root@mx86-bgl-1-r1#
>> root@mx86-bgl-1-r1# run ping 10.1.1.101
>> PING 10.1.1.101 (10.1.1.101): 56 data bytes
>> 64 bytes from 10.1.1.101: icmp_seq=0 ttl=64 time=0.980 ms
>> 64 bytes from 10.1.1.101: icmp_seq=1 ttl=64 time=1.433 ms
>>
>>
>> 3.  Default MAC address of interface ge-0/0/0  is as shown below:
>>
>> root@mx86-bgl-1-r1# run show interfaces ge-0/0/0 extensive | grep current
>>    Current address: 02:06:0a:0e:ff:f0, Hardware address: 02:06:0a:0e:ff:f0
>>
>> [edit]
>> root@mx86-bgl-1-r1#
>>
>>
>> 4.  Now I am changing MAC. Ping works this time also
>>
>> root@mx86-bgl-1-r1# set interfaces ge-0/0/0 mac 02:06:0a:0a:ff:f0
>>
>> [edit]
>> root@mx86-bgl-1-r1# commit
>> commit complete
>>
>> [edit]
>> root@mx86-bgl-1-r1# run show interfaces ge-0/0/0 extensive | grep current
>>    Current address: 02:06:0a:0a:ff:f0, Hardware address: 02:06:0a:0e:ff:f0
>>
>> [edit]
>> root@mx86-bgl-1-r1# show interfaces
>> ge-0/0/0 {
>>      mac 02:06:0a:0a:ff:f0;
>>      unit 0 {
>>          family inet {
>>              address 10.1.1.103/24;
>>          }
>>      }
>> }
>>
>> [edit]
>> root@mx86-bgl-1-r1#
>> root@mx86-bgl-1-r1# run ping 10.1.1.101
>> PING 10.1.1.101 (10.1.1.101): 56 data bytes
>> 64 bytes from 10.1.1.101: icmp_seq=0 ttl=64 time=1.338 ms
>> 64 bytes from 10.1.1.101: icmp_seq=1 ttl=64 time=8.832 ms
>> 64 bytes from 10.1.1.101: icmp_seq=2 ttl=64 time=1.292 ms
>>
>>
>> 5.  I am deleting the above configured MAC.
>>       In this case newly configured MAC as shown above will be deleted
>> and Default MAC (02:06:0a:0e:ff:f0)
>>        will be applied.  Ping fails at this step.  "return statement
>> added by you is not allowing to configure default MAC.
>>
>>
>> [edit]
>> root@mx86-bgl-1-r1# delete interfaces ge-0/0/0 mac
>>
>> [edit]
>> root@mx86-bgl-1-r1# commit
>> commit complete
>>
>> [edit]
>> root@mx86-bgl-1-r1# show interfaces
>> ge-0/0/0 {
>>      unit 0 {
>>          family inet {
>>              address 10.1.1.103/24;
>>          }
>>      }
>> }
>>
>> [edit]
>>
>> root@mx86-bgl-1-r1# run show interfaces ge-0/0/0 extensive | grep
>> current     // Displays value from router's local database not from
>> NIC.
>>    Current address: 02:06:0a:0e:ff:f0, Hardware address: 02:06:0a:0e:ff:f0
>>
>> [edit]
>> root@mx86-bgl-1-r1# run ping 10.1.1.101
>> PING 10.1.1.101 (10.1.1.101): 56 data bytes
>> ^C
>> 2 packets transmitted, 0 packets received, 100% packet loss
>>
>> [edit]
>> root@mx86-bgl-1-r1#
>>
>>
>> 7. LOGs:    (I have added a debug log just before the return statement
>> that you place)
>>
>> Log o/p in failure case
>> ....
>> Santosh ixgbevf_add_mac_addr returning
>> ....
>>
>> code changes:
>> -----------------------------
>> ixgbevf_add_mac_addr(....)  {
>> ...
>>      if (memcmp(hw->mac.perm_addr, mac_addr, sizeof(struct ether_addr)) == 0) {
>>          PMD_DRV_LOG(DEBUG, "Existing MAC \n");
>>          printf("Santosh %s returning \n",__FUNCTION__);
>>          return;
>>      }
>>
>>
>> 8.  If I remove the above "return" statement, build the driver, loaded
>> in router and repeat the steps-2 to steps-5 of MAC add and MAC delete
>> on our router then ping passes.
>>
>> root@mx86-bgl-1-r1# run ping 10.1.1.101
>> PING 10.1.1.101 (10.1.1.101): 56 data bytes
>> 64 bytes from 10.1.1.101: icmp_seq=0 ttl=64 time=2.356 ms
>> 64 bytes from 10.1.1.101: icmp_seq=1 ttl=64 time=1.415 ms
>> 64 bytes from 10.1.1.101: icmp_seq=2 ttl=64 time=1.226 ms
>> ^C
>> --- 10.1.1.101 ping statistics ---
>> 3 packets transmitted, 3 packets received, 0% packet loss
>> round-trip min/avg/max/stddev = 1.226/1.666/2.356/0.494 ms
>>
>>
>> 9.  Please let me know whether is it  fine to remove that return
>> statement added by you in  "ixgbevf_add_mac_addr"  ?
>>
>>
>>
>> Thanks & Regards,
>> Santosh
>>
>> On Wed, Apr 20, 2016 at 3:31 AM, Ivan Boule <ivan.boule@6wind.com> wrote:
>>> Hi Santosh,
>>>
>>> I do not get exactly what you attempt to do on a VF.
>>> Are you first deleting the so-called permanent MAC address by a call to the
>>> function ixgbevf_remove_mac_addr() ? This operation is not allowed.
>>> Can you explain exactly the sequence of operations that are done, so that I
>>> can understand how the test (memcmp(hw->mac.perm_addr, mac_addr,
>>> sizeof(struct ether_addr)) == 0) in the function ixgbevf_add_mac_addr()
>>> prevents them to be successfully performed.
>>>
>>> Ivan
>>>
>>> PS : please, can you CC your emails to dev@dpdk.org
>>>
>>>
>>> 2016-04-19 17:01 GMT+02:00 santosh <santosh.iitg@gmail.com>:
>>>>
>>>> Hi Ivan,
>>>>
>>>> Your following code changes causing issue in Vmware environment.
>>>>
>>>> ----------------------------------- -------------------
>>>> ------------------------------
>>>> + /*
>>>> + * On a 82599 VF, adding again the same MAC addr is not an idempotent
>>>> + * operation. Trap this case to avoid exhausting the [very limited]
>>>> + * set of PF resources used to store VF MAC addresses.
>>>> + */
>>>> + if (memcmp(hw->mac.perm_addr, mac_addr, sizeof(struct ether_addr)) == 0)
>>>> + return;
>>>>    diag = ixgbevf_set_uc_addr_vf(hw, 2, mac_addr->addr_bytes);
>>>> --------------------------------------------------------------------
>>>> ------------- -------
>>>>
>>>>
>>>> Issue:
>>>> At CLI , we have provision to add /del MAC of an interface.
>>>> During MAC delete, existing MAC is deleted and default MAC is applied.
>>>> This default MAC is not being applied in VMware environment
>>>> successfully due to "return" statement
>>>> in your above code changes. As a result traffic is stopped completely.
>>>> If I remove above
>>>> "return" statement then traffic continues to flow after MAC delete.
>>>>
>>>> Please let me know your suggestion to handle this scenario .  If I
>>>> remove "return" what will be the consequences ?
>>>>
>>>> If removing "return" statement is not good idea then what are other
>>>> way to handle MAC delete scenario ?  we have only 1 VF per PF in our
>>>> setup as of now.
>>>>
>>>>
>>>> Thanks
>>>> Santosh
>>>
>>>
>>>
>>>
>>> --
>>> Ivan BOULE
>>> 6WIND
>>> Software Engineer
>>>
>>> Tel: +33 1 39 30 92 47
>>> Mob: + 33 6 77 25 26 38
>>> Fax: +33 1 39 30 92 11
>>> ivan.boule@6wind.com
>>> www.6wind.com
>>> Join the Multicore Packet Processing Forum:
>>> www.multicorepacketprocessing.com
>>>
>>> Ce courriel ainsi que toutes les pièces jointes, est uniquement destiné à
>>> son ou ses destinataires. Il contient des informations confidentielles qui
>>> sont la propriété de 6WIND. Toute révélation, distribution ou copie des
>>> informations qu'il contient est strictement interdite. Si vous avez reçu ce
>>> message par erreur, veuillez immédiatement le signaler à l'émetteur et
>>> détruire toutes les données reçues.
>>>
>>> This e-mail message, including any attachments, is for the sole use of the
>>> intended recipient(s) and contains information that is confidential and
>>> proprietary to 6WIND. All unauthorized review, use, disclosure or
>>> distribution is prohibited. If you are not the intended recipient, please
>>> contact the sender by reply e-mail and destroy all copies of the original
>>> message.
>>>


-- 
Ivan Boule
6WIND Development Engineer

Re: [dpdk-dev] ixgbe : query regarding your code changes for VF mac add

[santosh]

Hi Ivan

 ixgbevf_set_default_mac_addr()   could not find in our code base.
put traces at other places as suggested by you.
 Log at "eth_ixgbevf_dev_init"  never displayed
Rest logs displayed as shown below.  I re-build the  driver module and
loaded on our virtual router and rebooted the system.

1.
Logs at the time of boot up:
-----------------------------------------

INIT: Initializing NIC port 0 RX queue 0 ...
INIT: Initializing NIC port 0 TX queue 0 ...
Santosh ixgbevf_add_mac_addr portid=0 mac=00:50:56:A0:10:C2
.....
....

Santosh ixgbevf_add_mac_addr portid=0 mac=00:50:56:A0:10:C2
RUNTIME: Detected port  0 status changed to UP.

2.
a.
Configured  a new MAC at Virtual Router CLI

root@mx86-bgl-2-r1# set interfaces ge-0/0/0 mac 00:50:56:a0:a0:c3

[edit]
root@mx86-bgl-2-r1# commit
commit complete

[edit]
root@mx86-bgl-2-r1# show interfaces
ge-0/0/0 {
    mac 00:50:56:a0:a0:c3;
    unit 0 {
        family inet {
            address 10.1.1.102/24;
        }
    }
}

[edit]
root@mx86-bgl-2-r1# run ping 10.1.1.101
PING 10.1.1.101 (10.1.1.101): 56 data bytes
64 bytes from 10.1.1.101: icmp_seq=0 ttl=64 time=2.142 ms
64 bytes from 10.1.1.101: icmp_seq=1 ttl=64 time=8.465 ms
^C
--- 10.1.1.101 ping statistics ---
2 packets transmitted, 2 packets received, 0% packet loss
round-trip min/avg/max/stddev = 2.142/5.303/8.465/3.162 ms

[edit]
root@mx86-bgl-2-r1#

b.
   Logs at console for above config:
   -------------------------------------------------

root@localhost:/home/pfe# Santosh ixgbevf_add_mac_addr portid=0
mac=00:50:56:A0:A0:C3

root@localhost:/home/pfe#


3.
a. Deleted above config MAC
root@mx86-bgl-2-r1# delete interfaces ge-0/0/0 mac

[edit]
root@mx86-bgl-2-r1# commit
commit complete

[edit]
root@mx86-bgl-2-r1# show interfaces
ge-0/0/0 {
    unit 0 {
        family inet {
            address 10.1.1.102/24;
        }
    }
}

[edit]
root@mx86-bgl-2-r1# run ping 10.1.1.101
PING 10.1.1.101 (10.1.1.101): 56 data bytes
^C
--- 10.1.1.101 ping statistics ---
3 packets transmitted, 0 packets received, 100% packet loss

[edit]
root@mx86-bgl-2-r1#

b.
 Logs at console for above cofig
 -------------------------


root@localhost:/home/pfe# ixgbevf_remove_mac_addr santosh portid=0
index=1 mac=00:50:56:A0:A0:C3



Thanks
Santosh














On Fri, Apr 22, 2016 at 12:51 PM, Ivan Boule <ivan.boule@6wind.com> wrote:
> Hi Santosh,
>
> My job at 6WIND does not consist in answering to DPDK questions. In general,
> I have other priorities, including vacations...
> In the meantime, nobody prevents you to add traces in the code to really
> understand what happens, as suggested in my last answer.
>
> Regards,
> Ivan
>
>
> On 04/21/2016 07:55 AM, santosh wrote:
>>
>> Hi Ivan and team,
>>
>> Please respond to my last mail and  let me know if there is any
>> alternate way to handle this.
>> Our release is in pending due to this issue.
>>
>>
>> Thanks & Regards
>> Santosh
>>
>> On Wed, Apr 20, 2016 at 2:35 PM, santosh <santosh.iitg@gmail.com> wrote:
>>>
>>> Hi Ivan,
>>>
>>> Thanks for your response.
>>>
>>> Let me explain you the issue that we are facing on our virtual router
>>> in VMware environment.
>>>
>>> 1. We are using ixgbe driver , SRIOV enabled .
>>>      root@localhost:~# lspci
>>>        .... "Intel Corporation 82599 Ethernet Controller Virtual
>>> Function"
>>>
>>> 2.  "mx86-bgl-1-r1"  is our router under testing and  R2 is a standard
>>> router.
>>>
>>>        mx86-bgl-1-r1 is connected to R2
>>>        mx86-bgl-1-r1 (10.1.1.103)------------------>R2(10.1.1.101)
>>>
>>> 2. This time ping test passes
>>>
>>> root@mx86-bgl-1-r1# show interfaces
>>> ge-0/0/0 {
>>>      unit 0 {
>>>          family inet {
>>>              address 10.1.1.103/24;
>>>          }
>>>      }
>>> }
>>>
>>> [edit]
>>> root@mx86-bgl-1-r1#
>>> root@mx86-bgl-1-r1# run ping 10.1.1.101
>>> PING 10.1.1.101 (10.1.1.101): 56 data bytes
>>> 64 bytes from 10.1.1.101: icmp_seq=0 ttl=64 time=0.980 ms
>>> 64 bytes from 10.1.1.101: icmp_seq=1 ttl=64 time=1.433 ms
>>>
>>>
>>> 3.  Default MAC address of interface ge-0/0/0  is as shown below:
>>>
>>> root@mx86-bgl-1-r1# run show interfaces ge-0/0/0 extensive | grep current
>>>    Current address: 02:06:0a:0e:ff:f0, Hardware address:
>>> 02:06:0a:0e:ff:f0
>>>
>>> [edit]
>>> root@mx86-bgl-1-r1#
>>>
>>>
>>> 4.  Now I am changing MAC. Ping works this time also
>>>
>>> root@mx86-bgl-1-r1# set interfaces ge-0/0/0 mac 02:06:0a:0a:ff:f0
>>>
>>> [edit]
>>> root@mx86-bgl-1-r1# commit
>>> commit complete
>>>
>>> [edit]
>>> root@mx86-bgl-1-r1# run show interfaces ge-0/0/0 extensive | grep current
>>>    Current address: 02:06:0a:0a:ff:f0, Hardware address:
>>> 02:06:0a:0e:ff:f0
>>>
>>> [edit]
>>> root@mx86-bgl-1-r1# show interfaces
>>> ge-0/0/0 {
>>>      mac 02:06:0a:0a:ff:f0;
>>>      unit 0 {
>>>          family inet {
>>>              address 10.1.1.103/24;
>>>          }
>>>      }
>>> }
>>>
>>> [edit]
>>> root@mx86-bgl-1-r1#
>>> root@mx86-bgl-1-r1# run ping 10.1.1.101
>>> PING 10.1.1.101 (10.1.1.101): 56 data bytes
>>> 64 bytes from 10.1.1.101: icmp_seq=0 ttl=64 time=1.338 ms
>>> 64 bytes from 10.1.1.101: icmp_seq=1 ttl=64 time=8.832 ms
>>> 64 bytes from 10.1.1.101: icmp_seq=2 ttl=64 time=1.292 ms
>>>
>>>
>>> 5.  I am deleting the above configured MAC.
>>>       In this case newly configured MAC as shown above will be deleted
>>> and Default MAC (02:06:0a:0e:ff:f0)
>>>        will be applied.  Ping fails at this step.  "return statement
>>> added by you is not allowing to configure default MAC.
>>>
>>>
>>> [edit]
>>> root@mx86-bgl-1-r1# delete interfaces ge-0/0/0 mac
>>>
>>> [edit]
>>> root@mx86-bgl-1-r1# commit
>>> commit complete
>>>
>>> [edit]
>>> root@mx86-bgl-1-r1# show interfaces
>>> ge-0/0/0 {
>>>      unit 0 {
>>>          family inet {
>>>              address 10.1.1.103/24;
>>>          }
>>>      }
>>> }
>>>
>>> [edit]
>>>
>>> root@mx86-bgl-1-r1# run show interfaces ge-0/0/0 extensive | grep
>>> current     // Displays value from router's local database not from
>>> NIC.
>>>    Current address: 02:06:0a:0e:ff:f0, Hardware address:
>>> 02:06:0a:0e:ff:f0
>>>
>>> [edit]
>>> root@mx86-bgl-1-r1# run ping 10.1.1.101
>>> PING 10.1.1.101 (10.1.1.101): 56 data bytes
>>> ^C
>>> 2 packets transmitted, 0 packets received, 100% packet loss
>>>
>>> [edit]
>>> root@mx86-bgl-1-r1#
>>>
>>>
>>> 7. LOGs:    (I have added a debug log just before the return statement
>>> that you place)
>>>
>>> Log o/p in failure case
>>> ....
>>> Santosh ixgbevf_add_mac_addr returning
>>> ....
>>>
>>> code changes:
>>> -----------------------------
>>> ixgbevf_add_mac_addr(....)  {
>>> ...
>>>      if (memcmp(hw->mac.perm_addr, mac_addr, sizeof(struct ether_addr))
>>> == 0) {
>>>          PMD_DRV_LOG(DEBUG, "Existing MAC \n");
>>>          printf("Santosh %s returning \n",__FUNCTION__);
>>>          return;
>>>      }
>>>
>>>
>>> 8.  If I remove the above "return" statement, build the driver, loaded
>>> in router and repeat the steps-2 to steps-5 of MAC add and MAC delete
>>> on our router then ping passes.
>>>
>>> root@mx86-bgl-1-r1# run ping 10.1.1.101
>>> PING 10.1.1.101 (10.1.1.101): 56 data bytes
>>> 64 bytes from 10.1.1.101: icmp_seq=0 ttl=64 time=2.356 ms
>>> 64 bytes from 10.1.1.101: icmp_seq=1 ttl=64 time=1.415 ms
>>> 64 bytes from 10.1.1.101: icmp_seq=2 ttl=64 time=1.226 ms
>>> ^C
>>> --- 10.1.1.101 ping statistics ---
>>> 3 packets transmitted, 3 packets received, 0% packet loss
>>> round-trip min/avg/max/stddev = 1.226/1.666/2.356/0.494 ms
>>>
>>>
>>> 9.  Please let me know whether is it  fine to remove that return
>>> statement added by you in  "ixgbevf_add_mac_addr"  ?
>>>
>>>
>>>
>>> Thanks & Regards,
>>> Santosh
>>>
>>> On Wed, Apr 20, 2016 at 3:31 AM, Ivan Boule <ivan.boule@6wind.com> wrote:
>>>>
>>>> Hi Santosh,
>>>>
>>>> I do not get exactly what you attempt to do on a VF.
>>>> Are you first deleting the so-called permanent MAC address by a call to
>>>> the
>>>> function ixgbevf_remove_mac_addr() ? This operation is not allowed.
>>>> Can you explain exactly the sequence of operations that are done, so
>>>> that I
>>>> can understand how the test (memcmp(hw->mac.perm_addr, mac_addr,
>>>> sizeof(struct ether_addr)) == 0) in the function ixgbevf_add_mac_addr()
>>>> prevents them to be successfully performed.
>>>>
>>>> Ivan
>>>>
>>>> PS : please, can you CC your emails to dev@dpdk.org
>>>>
>>>>
>>>> 2016-04-19 17:01 GMT+02:00 santosh <santosh.iitg@gmail.com>:
>>>>>
>>>>>
>>>>> Hi Ivan,
>>>>>
>>>>> Your following code changes causing issue in Vmware environment.
>>>>>
>>>>> ----------------------------------- -------------------
>>>>> ------------------------------
>>>>> + /*
>>>>> + * On a 82599 VF, adding again the same MAC addr is not an idempotent
>>>>> + * operation. Trap this case to avoid exhausting the [very limited]
>>>>> + * set of PF resources used to store VF MAC addresses.
>>>>> + */
>>>>> + if (memcmp(hw->mac.perm_addr, mac_addr, sizeof(struct ether_addr)) ==
>>>>> 0)
>>>>> + return;
>>>>>    diag = ixgbevf_set_uc_addr_vf(hw, 2, mac_addr->addr_bytes);
>>>>> --------------------------------------------------------------------
>>>>> ------------- -------
>>>>>
>>>>>
>>>>> Issue:
>>>>> At CLI , we have provision to add /del MAC of an interface.
>>>>> During MAC delete, existing MAC is deleted and default MAC is applied.
>>>>> This default MAC is not being applied in VMware environment
>>>>> successfully due to "return" statement
>>>>> in your above code changes. As a result traffic is stopped completely.
>>>>> If I remove above
>>>>> "return" statement then traffic continues to flow after MAC delete.
>>>>>
>>>>> Please let me know your suggestion to handle this scenario .  If I
>>>>> remove "return" what will be the consequences ?
>>>>>
>>>>> If removing "return" statement is not good idea then what are other
>>>>> way to handle MAC delete scenario ?  we have only 1 VF per PF in our
>>>>> setup as of now.
>>>>>
>>>>>
>>>>> Thanks
>>>>> Santosh
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> Ivan BOULE
>>>> 6WIND
>>>> Software Engineer
>>>>
>>>> Tel: +33 1 39 30 92 47
>>>> Mob: + 33 6 77 25 26 38
>>>> Fax: +33 1 39 30 92 11
>>>> ivan.boule@6wind.com
>>>> www.6wind.com
>>>> Join the Multicore Packet Processing Forum:
>>>> www.multicorepacketprocessing.com
>>>>
>>>> Ce courriel ainsi que toutes les pièces jointes, est uniquement destiné
>>>> à
>>>> son ou ses destinataires. Il contient des informations confidentielles
>>>> qui
>>>> sont la propriété de 6WIND. Toute révélation, distribution ou copie des
>>>> informations qu'il contient est strictement interdite. Si vous avez reçu
>>>> ce
>>>> message par erreur, veuillez immédiatement le signaler à l'émetteur et
>>>> détruire toutes les données reçues.
>>>>
>>>> This e-mail message, including any attachments, is for the sole use of
>>>> the
>>>> intended recipient(s) and contains information that is confidential and
>>>> proprietary to 6WIND. All unauthorized review, use, disclosure or
>>>> distribution is prohibited. If you are not the intended recipient,
>>>> please
>>>> contact the sender by reply e-mail and destroy all copies of the
>>>> original
>>>> message.
>>>>
>
>
> --
> Ivan Boule
> 6WIND Development Engineer

Re: [dpdk-dev] ixgbe : query regarding your code changes for VF mac add

[santosh]

Hi

Looks like there is a bug in API "ixgbevf_remove_mac_addr()".   For
deleting a given MAC it deletes all MAC (including permanent MAC) and
while adding (it does after few statements), it skips to add permanent
MAC.

ixgbevf_remove_mac_addr(struct rte_eth_dev *dev, uint32_t index)
{
.......

    /*
     * The IXGBE_VF_SET_MACVLAN command of the ixgbe-pf driver does
     * not support the deletion of a given MAC address.
     * Instead, it imposes to delete all MAC addresses, then to add again
     * all MAC addresses with the exception of the one to be deleted.
     */


    (void) ixgbevf_set_uc_addr_vf(hw, 0, NULL);
...

2.   Following modification at "ixgbevf_remove_mac_addr()}  also
helped to make ping works

$ git diff  dpdk/lib/librte_pmd_ixgbe/ixgbe_ethdev.c
--- a/dpdk/lib/librte_pmd_ixgbe/ixgbe_ethdev.c
+++ b/dpdk/lib/librte_pmd_ixgbe/ixgbe_ethdev.c

         @@ -3530,10 +3557,7 @@ ixgbevf_remove_mac_addr(struct
rte_eth_dev *dev, uint32_t index)
                /* Skip NULL MAC addresses */
                if (is_zero_ether_addr(mac_addr))
                        continue;
-               /* Skip the permanent MAC address */
-               if (memcmp(perm_addr, mac_addr, sizeof(struct ether_addr)) == 0)
-                       continue;

On Mon, Apr 25, 2016 at 7:05 PM, santosh <santosh.iitg@gmail.com> wrote:
> Hi Ivan
>
>  ixgbevf_set_default_mac_addr()   could not find in our code base.
> put traces at other places as suggested by you.
>  Log at "eth_ixgbevf_dev_init"  never displayed
> Rest logs displayed as shown below.  I re-build the  driver module and
> loaded on our virtual router and rebooted the system.
>
> 1.
> Logs at the time of boot up:
> -----------------------------------------
>
> INIT: Initializing NIC port 0 RX queue 0 ...
> INIT: Initializing NIC port 0 TX queue 0 ...
> Santosh ixgbevf_add_mac_addr portid=0 mac=00:50:56:A0:10:C2
> .....
> ....
>
> Santosh ixgbevf_add_mac_addr portid=0 mac=00:50:56:A0:10:C2
> RUNTIME: Detected port  0 status changed to UP.
>
> 2.
> a.
> Configured  a new MAC at Virtual Router CLI
>
> root@mx86-bgl-2-r1# set interfaces ge-0/0/0 mac 00:50:56:a0:a0:c3
>
> [edit]
> root@mx86-bgl-2-r1# commit
> commit complete
>
> [edit]
> root@mx86-bgl-2-r1# show interfaces
> ge-0/0/0 {
>     mac 00:50:56:a0:a0:c3;
>     unit 0 {
>         family inet {
>             address 10.1.1.102/24;
>         }
>     }
> }
>
> [edit]
> root@mx86-bgl-2-r1# run ping 10.1.1.101
> PING 10.1.1.101 (10.1.1.101): 56 data bytes
> 64 bytes from 10.1.1.101: icmp_seq=0 ttl=64 time=2.142 ms
> 64 bytes from 10.1.1.101: icmp_seq=1 ttl=64 time=8.465 ms
> ^C
> --- 10.1.1.101 ping statistics ---
> 2 packets transmitted, 2 packets received, 0% packet loss
> round-trip min/avg/max/stddev = 2.142/5.303/8.465/3.162 ms
>
> [edit]
> root@mx86-bgl-2-r1#
>
> b.
>    Logs at console for above config:
>    -------------------------------------------------
>
> root@localhost:/home/pfe# Santosh ixgbevf_add_mac_addr portid=0
> mac=00:50:56:A0:A0:C3
>
> root@localhost:/home/pfe#
>
>
> 3.
> a. Deleted above config MAC
> root@mx86-bgl-2-r1# delete interfaces ge-0/0/0 mac
>
> [edit]
> root@mx86-bgl-2-r1# commit
> commit complete
>
> [edit]
> root@mx86-bgl-2-r1# show interfaces
> ge-0/0/0 {
>     unit 0 {
>         family inet {
>             address 10.1.1.102/24;
>         }
>     }
> }
>
> [edit]
> root@mx86-bgl-2-r1# run ping 10.1.1.101
> PING 10.1.1.101 (10.1.1.101): 56 data bytes
> ^C
> --- 10.1.1.101 ping statistics ---
> 3 packets transmitted, 0 packets received, 100% packet loss
>
> [edit]
> root@mx86-bgl-2-r1#
>
> b.
>  Logs at console for above cofig
>  -------------------------
>
>
> root@localhost:/home/pfe# ixgbevf_remove_mac_addr santosh portid=0
> index=1 mac=00:50:56:A0:A0:C3
>
>
>
> Thanks
> Santosh
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> On Fri, Apr 22, 2016 at 12:51 PM, Ivan Boule <ivan.boule@6wind.com> wrote:
>> Hi Santosh,
>>
>> My job at 6WIND does not consist in answering to DPDK questions. In general,
>> I have other priorities, including vacations...
>> In the meantime, nobody prevents you to add traces in the code to really
>> understand what happens, as suggested in my last answer.
>>
>> Regards,
>> Ivan
>>
>>
>> On 04/21/2016 07:55 AM, santosh wrote:
>>>
>>> Hi Ivan and team,
>>>
>>> Please respond to my last mail and  let me know if there is any
>>> alternate way to handle this.
>>> Our release is in pending due to this issue.
>>>
>>>
>>> Thanks & Regards
>>> Santosh
>>>
>>> On Wed, Apr 20, 2016 at 2:35 PM, santosh <santosh.iitg@gmail.com> wrote:
>>>>
>>>> Hi Ivan,
>>>>
>>>> Thanks for your response.
>>>>
>>>> Let me explain you the issue that we are facing on our virtual router
>>>> in VMware environment.
>>>>
>>>> 1. We are using ixgbe driver , SRIOV enabled .
>>>>      root@localhost:~# lspci
>>>>        .... "Intel Corporation 82599 Ethernet Controller Virtual
>>>> Function"
>>>>
>>>> 2.  "mx86-bgl-1-r1"  is our router under testing and  R2 is a standard
>>>> router.
>>>>
>>>>        mx86-bgl-1-r1 is connected to R2
>>>>        mx86-bgl-1-r1 (10.1.1.103)------------------>R2(10.1.1.101)
>>>>
>>>> 2. This time ping test passes
>>>>
>>>> root@mx86-bgl-1-r1# show interfaces
>>>> ge-0/0/0 {
>>>>      unit 0 {
>>>>          family inet {
>>>>              address 10.1.1.103/24;
>>>>          }
>>>>      }
>>>> }
>>>>
>>>> [edit]
>>>> root@mx86-bgl-1-r1#
>>>> root@mx86-bgl-1-r1# run ping 10.1.1.101
>>>> PING 10.1.1.101 (10.1.1.101): 56 data bytes
>>>> 64 bytes from 10.1.1.101: icmp_seq=0 ttl=64 time=0.980 ms
>>>> 64 bytes from 10.1.1.101: icmp_seq=1 ttl=64 time=1.433 ms
>>>>
>>>>
>>>> 3.  Default MAC address of interface ge-0/0/0  is as shown below:
>>>>
>>>> root@mx86-bgl-1-r1# run show interfaces ge-0/0/0 extensive | grep current
>>>>    Current address: 02:06:0a:0e:ff:f0, Hardware address:
>>>> 02:06:0a:0e:ff:f0
>>>>
>>>> [edit]
>>>> root@mx86-bgl-1-r1#
>>>>
>>>>
>>>> 4.  Now I am changing MAC. Ping works this time also
>>>>
>>>> root@mx86-bgl-1-r1# set interfaces ge-0/0/0 mac 02:06:0a:0a:ff:f0
>>>>
>>>> [edit]
>>>> root@mx86-bgl-1-r1# commit
>>>> commit complete
>>>>
>>>> [edit]
>>>> root@mx86-bgl-1-r1# run show interfaces ge-0/0/0 extensive | grep current
>>>>    Current address: 02:06:0a:0a:ff:f0, Hardware address:
>>>> 02:06:0a:0e:ff:f0
>>>>
>>>> [edit]
>>>> root@mx86-bgl-1-r1# show interfaces
>>>> ge-0/0/0 {
>>>>      mac 02:06:0a:0a:ff:f0;
>>>>      unit 0 {
>>>>          family inet {
>>>>              address 10.1.1.103/24;
>>>>          }
>>>>      }
>>>> }
>>>>
>>>> [edit]
>>>> root@mx86-bgl-1-r1#
>>>> root@mx86-bgl-1-r1# run ping 10.1.1.101
>>>> PING 10.1.1.101 (10.1.1.101): 56 data bytes
>>>> 64 bytes from 10.1.1.101: icmp_seq=0 ttl=64 time=1.338 ms
>>>> 64 bytes from 10.1.1.101: icmp_seq=1 ttl=64 time=8.832 ms
>>>> 64 bytes from 10.1.1.101: icmp_seq=2 ttl=64 time=1.292 ms
>>>>
>>>>
>>>> 5.  I am deleting the above configured MAC.
>>>>       In this case newly configured MAC as shown above will be deleted
>>>> and Default MAC (02:06:0a:0e:ff:f0)
>>>>        will be applied.  Ping fails at this step.  "return statement
>>>> added by you is not allowing to configure default MAC.
>>>>
>>>>
>>>> [edit]
>>>> root@mx86-bgl-1-r1# delete interfaces ge-0/0/0 mac
>>>>
>>>> [edit]
>>>> root@mx86-bgl-1-r1# commit
>>>> commit complete
>>>>
>>>> [edit]
>>>> root@mx86-bgl-1-r1# show interfaces
>>>> ge-0/0/0 {
>>>>      unit 0 {
>>>>          family inet {
>>>>              address 10.1.1.103/24;
>>>>          }
>>>>      }
>>>> }
>>>>
>>>> [edit]
>>>>
>>>> root@mx86-bgl-1-r1# run show interfaces ge-0/0/0 extensive | grep
>>>> current     // Displays value from router's local database not from
>>>> NIC.
>>>>    Current address: 02:06:0a:0e:ff:f0, Hardware address:
>>>> 02:06:0a:0e:ff:f0
>>>>
>>>> [edit]
>>>> root@mx86-bgl-1-r1# run ping 10.1.1.101
>>>> PING 10.1.1.101 (10.1.1.101): 56 data bytes
>>>> ^C
>>>> 2 packets transmitted, 0 packets received, 100% packet loss
>>>>
>>>> [edit]
>>>> root@mx86-bgl-1-r1#
>>>>
>>>>
>>>> 7. LOGs:    (I have added a debug log just before the return statement
>>>> that you place)
>>>>
>>>> Log o/p in failure case
>>>> ....
>>>> Santosh ixgbevf_add_mac_addr returning
>>>> ....
>>>>
>>>> code changes:
>>>> -----------------------------
>>>> ixgbevf_add_mac_addr(....)  {
>>>> ...
>>>>      if (memcmp(hw->mac.perm_addr, mac_addr, sizeof(struct ether_addr))
>>>> == 0) {
>>>>          PMD_DRV_LOG(DEBUG, "Existing MAC \n");
>>>>          printf("Santosh %s returning \n",__FUNCTION__);
>>>>          return;
>>>>      }
>>>>
>>>>
>>>> 8.  If I remove the above "return" statement, build the driver, loaded
>>>> in router and repeat the steps-2 to steps-5 of MAC add and MAC delete
>>>> on our router then ping passes.
>>>>
>>>> root@mx86-bgl-1-r1# run ping 10.1.1.101
>>>> PING 10.1.1.101 (10.1.1.101): 56 data bytes
>>>> 64 bytes from 10.1.1.101: icmp_seq=0 ttl=64 time=2.356 ms
>>>> 64 bytes from 10.1.1.101: icmp_seq=1 ttl=64 time=1.415 ms
>>>> 64 bytes from 10.1.1.101: icmp_seq=2 ttl=64 time=1.226 ms
>>>> ^C
>>>> --- 10.1.1.101 ping statistics ---
>>>> 3 packets transmitted, 3 packets received, 0% packet loss
>>>> round-trip min/avg/max/stddev = 1.226/1.666/2.356/0.494 ms
>>>>
>>>>
>>>> 9.  Please let me know whether is it  fine to remove that return
>>>> statement added by you in  "ixgbevf_add_mac_addr"  ?
>>>>
>>>>
>>>>
>>>> Thanks & Regards,
>>>> Santosh
>>>>
>>>> On Wed, Apr 20, 2016 at 3:31 AM, Ivan Boule <ivan.boule@6wind.com> wrote:
>>>>>
>>>>> Hi Santosh,
>>>>>
>>>>> I do not get exactly what you attempt to do on a VF.
>>>>> Are you first deleting the so-called permanent MAC address by a call to
>>>>> the
>>>>> function ixgbevf_remove_mac_addr() ? This operation is not allowed.
>>>>> Can you explain exactly the sequence of operations that are done, so
>>>>> that I
>>>>> can understand how the test (memcmp(hw->mac.perm_addr, mac_addr,
>>>>> sizeof(struct ether_addr)) == 0) in the function ixgbevf_add_mac_addr()
>>>>> prevents them to be successfully performed.
>>>>>
>>>>> Ivan
>>>>>
>>>>> PS : please, can you CC your emails to dev@dpdk.org
>>>>>
>>>>>
>>>>> 2016-04-19 17:01 GMT+02:00 santosh <santosh.iitg@gmail.com>:
>>>>>>
>>>>>>
>>>>>> Hi Ivan,
>>>>>>
>>>>>> Your following code changes causing issue in Vmware environment.
>>>>>>
>>>>>> ----------------------------------- -------------------
>>>>>> ------------------------------
>>>>>> + /*
>>>>>> + * On a 82599 VF, adding again the same MAC addr is not an idempotent
>>>>>> + * operation. Trap this case to avoid exhausting the [very limited]
>>>>>> + * set of PF resources used to store VF MAC addresses.
>>>>>> + */
>>>>>> + if (memcmp(hw->mac.perm_addr, mac_addr, sizeof(struct ether_addr)) ==
>>>>>> 0)
>>>>>> + return;
>>>>>>    diag = ixgbevf_set_uc_addr_vf(hw, 2, mac_addr->addr_bytes);
>>>>>> --------------------------------------------------------------------
>>>>>> ------------- -------
>>>>>>
>>>>>>
>>>>>> Issue:
>>>>>> At CLI , we have provision to add /del MAC of an interface.
>>>>>> During MAC delete, existing MAC is deleted and default MAC is applied.
>>>>>> This default MAC is not being applied in VMware environment
>>>>>> successfully due to "return" statement
>>>>>> in your above code changes. As a result traffic is stopped completely.
>>>>>> If I remove above
>>>>>> "return" statement then traffic continues to flow after MAC delete.
>>>>>>
>>>>>> Please let me know your suggestion to handle this scenario .  If I
>>>>>> remove "return" what will be the consequences ?
>>>>>>
>>>>>> If removing "return" statement is not good idea then what are other
>>>>>> way to handle MAC delete scenario ?  we have only 1 VF per PF in our
>>>>>> setup as of now.
>>>>>>
>>>>>>
>>>>>> Thanks
>>>>>> Santosh
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Ivan BOULE
>>>>> 6WIND
>>>>> Software Engineer
>>>>>
>>>>> Tel: +33 1 39 30 92 47
>>>>> Mob: + 33 6 77 25 26 38
>>>>> Fax: +33 1 39 30 92 11
>>>>> ivan.boule@6wind.com
>>>>> www.6wind.com
>>>>> Join the Multicore Packet Processing Forum:
>>>>> www.multicorepacketprocessing.com
>>>>>
>>>>> Ce courriel ainsi que toutes les pièces jointes, est uniquement destiné
>>>>> à
>>>>> son ou ses destinataires. Il contient des informations confidentielles
>>>>> qui
>>>>> sont la propriété de 6WIND. Toute révélation, distribution ou copie des
>>>>> informations qu'il contient est strictement interdite. Si vous avez reçu
>>>>> ce
>>>>> message par erreur, veuillez immédiatement le signaler à l'émetteur et
>>>>> détruire toutes les données reçues.
>>>>>
>>>>> This e-mail message, including any attachments, is for the sole use of
>>>>> the
>>>>> intended recipient(s) and contains information that is confidential and
>>>>> proprietary to 6WIND. All unauthorized review, use, disclosure or
>>>>> distribution is prohibited. If you are not the intended recipient,
>>>>> please
>>>>> contact the sender by reply e-mail and destroy all copies of the
>>>>> original
>>>>> message.
>>>>>
>>
>>
>> --
>> Ivan Boule
>> 6WIND Development Engineer

Re: [dpdk-dev] ixgbe : query regarding your code changes for VF mac add

[Ivan Boule]

Hi Santosh,

Things are a little bit more complex.
When the permanent MAC address of a VF is assigned by the PF driver, it 
cannot be changed nor deleted by the VF through commands issued to the 
PF driver. In this case, the function xgbevf_remove_mac_addr() must not 
add the permanent MAC address again, as currently done.

Conversely, when the permanent MAC address of a VF is assigned by the Vf 
itself, then it will be also deleted by the call to 
ixgbevf_set_uc_addr_vf(hw, 0, NULL) in the function 
xgbevf_remove_mac_addr (as other MAC addresses that have been previously 
added through a call to the function ixgbevf_add_mac_addr).
In this case, the function xgbevf_remove_mac_addr must effectively add 
the permanent MAC address again.

To resume, the ixgbevf PMD must record in a new dedicated flag if the 
permanent MAC address of a VF has been assigned by the PF driver or not, 
and test this flag in the function xgbevf_remove_mac_addr to decide if 
it must add again the permanent MAC address of the VF.

Regards,
Ivan

On 04/26/2016 07:36 AM, santosh wrote:
> Hi
>
> Looks like there is a bug in API "ixgbevf_remove_mac_addr()".   For
> deleting a given MAC it deletes all MAC (including permanent MAC) and
> while adding (it does after few statements), it skips to add permanent
> MAC.
>
> ixgbevf_remove_mac_addr(struct rte_eth_dev *dev, uint32_t index)
> {
> .......
>
>      /*
>       * The IXGBE_VF_SET_MACVLAN command of the ixgbe-pf driver does
>       * not support the deletion of a given MAC address.
>       * Instead, it imposes to delete all MAC addresses, then to add again
>       * all MAC addresses with the exception of the one to be deleted.
>       */
>
>
>      (void) ixgbevf_set_uc_addr_vf(hw, 0, NULL);
> ...
>
> 2.   Following modification at "ixgbevf_remove_mac_addr()}  also
> helped to make ping works
>
> $ git diff  dpdk/lib/librte_pmd_ixgbe/ixgbe_ethdev.c
> --- a/dpdk/lib/librte_pmd_ixgbe/ixgbe_ethdev.c
> +++ b/dpdk/lib/librte_pmd_ixgbe/ixgbe_ethdev.c
>
>           @@ -3530,10 +3557,7 @@ ixgbevf_remove_mac_addr(struct
> rte_eth_dev *dev, uint32_t index)
>                  /* Skip NULL MAC addresses */
>                  if (is_zero_ether_addr(mac_addr))
>                          continue;
> -               /* Skip the permanent MAC address */
> -               if (memcmp(perm_addr, mac_addr, sizeof(struct ether_addr)) == 0)
> -                       continue;
>
> On Mon, Apr 25, 2016 at 7:05 PM, santosh <santosh.iitg@gmail.com> wrote:
>> Hi Ivan
>>
>>   ixgbevf_set_default_mac_addr()   could not find in our code base.
>> put traces at other places as suggested by you.
>>   Log at "eth_ixgbevf_dev_init"  never displayed
>> Rest logs displayed as shown below.  I re-build the  driver module and
>> loaded on our virtual router and rebooted the system.
>>
>> 1.
>> Logs at the time of boot up:
>> -----------------------------------------
>>
>> INIT: Initializing NIC port 0 RX queue 0 ...
>> INIT: Initializing NIC port 0 TX queue 0 ...
>> Santosh ixgbevf_add_mac_addr portid=0 mac=00:50:56:A0:10:C2
>> .....
>> ....
>>
>> Santosh ixgbevf_add_mac_addr portid=0 mac=00:50:56:A0:10:C2
>> RUNTIME: Detected port  0 status changed to UP.
>>
>> 2.
>> a.
>> Configured  a new MAC at Virtual Router CLI
>>
>> root@mx86-bgl-2-r1# set interfaces ge-0/0/0 mac 00:50:56:a0:a0:c3
>>
>> [edit]
>> root@mx86-bgl-2-r1# commit
>> commit complete
>>
>> [edit]
>> root@mx86-bgl-2-r1# show interfaces
>> ge-0/0/0 {
>>      mac 00:50:56:a0:a0:c3;
>>      unit 0 {
>>          family inet {
>>              address 10.1.1.102/24;
>>          }
>>      }
>> }
>>
>> [edit]
>> root@mx86-bgl-2-r1# run ping 10.1.1.101
>> PING 10.1.1.101 (10.1.1.101): 56 data bytes
>> 64 bytes from 10.1.1.101: icmp_seq=0 ttl=64 time=2.142 ms
>> 64 bytes from 10.1.1.101: icmp_seq=1 ttl=64 time=8.465 ms
>> ^C
>> --- 10.1.1.101 ping statistics ---
>> 2 packets transmitted, 2 packets received, 0% packet loss
>> round-trip min/avg/max/stddev = 2.142/5.303/8.465/3.162 ms
>>
>> [edit]
>> root@mx86-bgl-2-r1#
>>
>> b.
>>     Logs at console for above config:
>>     -------------------------------------------------
>>
>> root@localhost:/home/pfe# Santosh ixgbevf_add_mac_addr portid=0
>> mac=00:50:56:A0:A0:C3
>>
>> root@localhost:/home/pfe#
>>
>>
>> 3.
>> a. Deleted above config MAC
>> root@mx86-bgl-2-r1# delete interfaces ge-0/0/0 mac
>>
>> [edit]
>> root@mx86-bgl-2-r1# commit
>> commit complete
>>
>> [edit]
>> root@mx86-bgl-2-r1# show interfaces
>> ge-0/0/0 {
>>      unit 0 {
>>          family inet {
>>              address 10.1.1.102/24;
>>          }
>>      }
>> }
>>
>> [edit]
>> root@mx86-bgl-2-r1# run ping 10.1.1.101
>> PING 10.1.1.101 (10.1.1.101): 56 data bytes
>> ^C
>> --- 10.1.1.101 ping statistics ---
>> 3 packets transmitted, 0 packets received, 100% packet loss
>>
>> [edit]
>> root@mx86-bgl-2-r1#
>>
>> b.
>>   Logs at console for above cofig
>>   -------------------------
>>
>>
>> root@localhost:/home/pfe# ixgbevf_remove_mac_addr santosh portid=0
>> index=1 mac=00:50:56:A0:A0:C3
>>
>>
>>
>> Thanks
>> Santosh
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>> On Fri, Apr 22, 2016 at 12:51 PM, Ivan Boule <ivan.boule@6wind.com> wrote:
>>> Hi Santosh,
>>>
>>> My job at 6WIND does not consist in answering to DPDK questions. In general,
>>> I have other priorities, including vacations...
>>> In the meantime, nobody prevents you to add traces in the code to really
>>> understand what happens, as suggested in my last answer.
>>>
>>> Regards,
>>> Ivan
>>>
>>>
>>> On 04/21/2016 07:55 AM, santosh wrote:
>>>>
>>>> Hi Ivan and team,
>>>>
>>>> Please respond to my last mail and  let me know if there is any
>>>> alternate way to handle this.
>>>> Our release is in pending due to this issue.
>>>>
>>>>
>>>> Thanks & Regards
>>>> Santosh
>>>>
>>>> On Wed, Apr 20, 2016 at 2:35 PM, santosh <santosh.iitg@gmail.com> wrote:
>>>>>
>>>>> Hi Ivan,
>>>>>
>>>>> Thanks for your response.
>>>>>
>>>>> Let me explain you the issue that we are facing on our virtual router
>>>>> in VMware environment.
>>>>>
>>>>> 1. We are using ixgbe driver , SRIOV enabled .
>>>>>       root@localhost:~# lspci
>>>>>         .... "Intel Corporation 82599 Ethernet Controller Virtual
>>>>> Function"
>>>>>
>>>>> 2.  "mx86-bgl-1-r1"  is our router under testing and  R2 is a standard
>>>>> router.
>>>>>
>>>>>         mx86-bgl-1-r1 is connected to R2
>>>>>         mx86-bgl-1-r1 (10.1.1.103)------------------>R2(10.1.1.101)
>>>>>
>>>>> 2. This time ping test passes
>>>>>
>>>>> root@mx86-bgl-1-r1# show interfaces
>>>>> ge-0/0/0 {
>>>>>       unit 0 {
>>>>>           family inet {
>>>>>               address 10.1.1.103/24;
>>>>>           }
>>>>>       }
>>>>> }
>>>>>
>>>>> [edit]
>>>>> root@mx86-bgl-1-r1#
>>>>> root@mx86-bgl-1-r1# run ping 10.1.1.101
>>>>> PING 10.1.1.101 (10.1.1.101): 56 data bytes
>>>>> 64 bytes from 10.1.1.101: icmp_seq=0 ttl=64 time=0.980 ms
>>>>> 64 bytes from 10.1.1.101: icmp_seq=1 ttl=64 time=1.433 ms
>>>>>
>>>>>
>>>>> 3.  Default MAC address of interface ge-0/0/0  is as shown below:
>>>>>
>>>>> root@mx86-bgl-1-r1# run show interfaces ge-0/0/0 extensive | grep current
>>>>>     Current address: 02:06:0a:0e:ff:f0, Hardware address:
>>>>> 02:06:0a:0e:ff:f0
>>>>>
>>>>> [edit]
>>>>> root@mx86-bgl-1-r1#
>>>>>
>>>>>
>>>>> 4.  Now I am changing MAC. Ping works this time also
>>>>>
>>>>> root@mx86-bgl-1-r1# set interfaces ge-0/0/0 mac 02:06:0a:0a:ff:f0
>>>>>
>>>>> [edit]
>>>>> root@mx86-bgl-1-r1# commit
>>>>> commit complete
>>>>>
>>>>> [edit]
>>>>> root@mx86-bgl-1-r1# run show interfaces ge-0/0/0 extensive | grep current
>>>>>     Current address: 02:06:0a:0a:ff:f0, Hardware address:
>>>>> 02:06:0a:0e:ff:f0
>>>>>
>>>>> [edit]
>>>>> root@mx86-bgl-1-r1# show interfaces
>>>>> ge-0/0/0 {
>>>>>       mac 02:06:0a:0a:ff:f0;
>>>>>       unit 0 {
>>>>>           family inet {
>>>>>               address 10.1.1.103/24;
>>>>>           }
>>>>>       }
>>>>> }
>>>>>
>>>>> [edit]
>>>>> root@mx86-bgl-1-r1#
>>>>> root@mx86-bgl-1-r1# run ping 10.1.1.101
>>>>> PING 10.1.1.101 (10.1.1.101): 56 data bytes
>>>>> 64 bytes from 10.1.1.101: icmp_seq=0 ttl=64 time=1.338 ms
>>>>> 64 bytes from 10.1.1.101: icmp_seq=1 ttl=64 time=8.832 ms
>>>>> 64 bytes from 10.1.1.101: icmp_seq=2 ttl=64 time=1.292 ms
>>>>>
>>>>>
>>>>> 5.  I am deleting the above configured MAC.
>>>>>        In this case newly configured MAC as shown above will be deleted
>>>>> and Default MAC (02:06:0a:0e:ff:f0)
>>>>>         will be applied.  Ping fails at this step.  "return statement
>>>>> added by you is not allowing to configure default MAC.
>>>>>
>>>>>
>>>>> [edit]
>>>>> root@mx86-bgl-1-r1# delete interfaces ge-0/0/0 mac
>>>>>
>>>>> [edit]
>>>>> root@mx86-bgl-1-r1# commit
>>>>> commit complete
>>>>>
>>>>> [edit]
>>>>> root@mx86-bgl-1-r1# show interfaces
>>>>> ge-0/0/0 {
>>>>>       unit 0 {
>>>>>           family inet {
>>>>>               address 10.1.1.103/24;
>>>>>           }
>>>>>       }
>>>>> }
>>>>>
>>>>> [edit]
>>>>>
>>>>> root@mx86-bgl-1-r1# run show interfaces ge-0/0/0 extensive | grep
>>>>> current     // Displays value from router's local database not from
>>>>> NIC.
>>>>>     Current address: 02:06:0a:0e:ff:f0, Hardware address:
>>>>> 02:06:0a:0e:ff:f0
>>>>>
>>>>> [edit]
>>>>> root@mx86-bgl-1-r1# run ping 10.1.1.101
>>>>> PING 10.1.1.101 (10.1.1.101): 56 data bytes
>>>>> ^C
>>>>> 2 packets transmitted, 0 packets received, 100% packet loss
>>>>>
>>>>> [edit]
>>>>> root@mx86-bgl-1-r1#
>>>>>
>>>>>
>>>>> 7. LOGs:    (I have added a debug log just before the return statement
>>>>> that you place)
>>>>>
>>>>> Log o/p in failure case
>>>>> ....
>>>>> Santosh ixgbevf_add_mac_addr returning
>>>>> ....
>>>>>
>>>>> code changes:
>>>>> -----------------------------
>>>>> ixgbevf_add_mac_addr(....)  {
>>>>> ...
>>>>>       if (memcmp(hw->mac.perm_addr, mac_addr, sizeof(struct ether_addr))
>>>>> == 0) {
>>>>>           PMD_DRV_LOG(DEBUG, "Existing MAC \n");
>>>>>           printf("Santosh %s returning \n",__FUNCTION__);
>>>>>           return;
>>>>>       }
>>>>>
>>>>>
>>>>> 8.  If I remove the above "return" statement, build the driver, loaded
>>>>> in router and repeat the steps-2 to steps-5 of MAC add and MAC delete
>>>>> on our router then ping passes.
>>>>>
>>>>> root@mx86-bgl-1-r1# run ping 10.1.1.101
>>>>> PING 10.1.1.101 (10.1.1.101): 56 data bytes
>>>>> 64 bytes from 10.1.1.101: icmp_seq=0 ttl=64 time=2.356 ms
>>>>> 64 bytes from 10.1.1.101: icmp_seq=1 ttl=64 time=1.415 ms
>>>>> 64 bytes from 10.1.1.101: icmp_seq=2 ttl=64 time=1.226 ms
>>>>> ^C
>>>>> --- 10.1.1.101 ping statistics ---
>>>>> 3 packets transmitted, 3 packets received, 0% packet loss
>>>>> round-trip min/avg/max/stddev = 1.226/1.666/2.356/0.494 ms
>>>>>
>>>>>
>>>>> 9.  Please let me know whether is it  fine to remove that return
>>>>> statement added by you in  "ixgbevf_add_mac_addr"  ?
>>>>>
>>>>>
>>>>>
>>>>> Thanks & Regards,
>>>>> Santosh
>>>>>
>>>>> On Wed, Apr 20, 2016 at 3:31 AM, Ivan Boule <ivan.boule@6wind.com> wrote:
>>>>>>
>>>>>> Hi Santosh,
>>>>>>
>>>>>> I do not get exactly what you attempt to do on a VF.
>>>>>> Are you first deleting the so-called permanent MAC address by a call to
>>>>>> the
>>>>>> function ixgbevf_remove_mac_addr() ? This operation is not allowed.
>>>>>> Can you explain exactly the sequence of operations that are done, so
>>>>>> that I
>>>>>> can understand how the test (memcmp(hw->mac.perm_addr, mac_addr,
>>>>>> sizeof(struct ether_addr)) == 0) in the function ixgbevf_add_mac_addr()
>>>>>> prevents them to be successfully performed.
>>>>>>
>>>>>> Ivan
>>>>>>
>>>>>> PS : please, can you CC your emails to dev@dpdk.org
>>>>>>
>>>>>>
>>>>>> 2016-04-19 17:01 GMT+02:00 santosh <santosh.iitg@gmail.com>:
>>>>>>>
>>>>>>>
>>>>>>> Hi Ivan,
>>>>>>>
>>>>>>> Your following code changes causing issue in Vmware environment.
>>>>>>>
>>>>>>> ----------------------------------- -------------------
>>>>>>> ------------------------------
>>>>>>> + /*
>>>>>>> + * On a 82599 VF, adding again the same MAC addr is not an idempotent
>>>>>>> + * operation. Trap this case to avoid exhausting the [very limited]
>>>>>>> + * set of PF resources used to store VF MAC addresses.
>>>>>>> + */
>>>>>>> + if (memcmp(hw->mac.perm_addr, mac_addr, sizeof(struct ether_addr)) ==
>>>>>>> 0)
>>>>>>> + return;
>>>>>>>     diag = ixgbevf_set_uc_addr_vf(hw, 2, mac_addr->addr_bytes);
>>>>>>> --------------------------------------------------------------------
>>>>>>> ------------- -------
>>>>>>>
>>>>>>>
>>>>>>> Issue:
>>>>>>> At CLI , we have provision to add /del MAC of an interface.
>>>>>>> During MAC delete, existing MAC is deleted and default MAC is applied.
>>>>>>> This default MAC is not being applied in VMware environment
>>>>>>> successfully due to "return" statement
>>>>>>> in your above code changes. As a result traffic is stopped completely.
>>>>>>> If I remove above
>>>>>>> "return" statement then traffic continues to flow after MAC delete.
>>>>>>>
>>>>>>> Please let me know your suggestion to handle this scenario .  If I
>>>>>>> remove "return" what will be the consequences ?
>>>>>>>
>>>>>>> If removing "return" statement is not good idea then what are other
>>>>>>> way to handle MAC delete scenario ?  we have only 1 VF per PF in our
>>>>>>> setup as of now.
>>>>>>>
>>>>>>>
>>>>>>> Thanks
>>>>>>> Santosh
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> Ivan BOULE
>>>>>> 6WIND
>>>>>> Software Engineer
>>>>>>
>>>>>> Tel: +33 1 39 30 92 47
>>>>>> Mob: + 33 6 77 25 26 38
>>>>>> Fax: +33 1 39 30 92 11
>>>>>> ivan.boule@6wind.com
>>>>>> www.6wind.com
>>>>>> Join the Multicore Packet Processing Forum:
>>>>>> www.multicorepacketprocessing.com
>>>>>>
>>>>>> Ce courriel ainsi que toutes les pièces jointes, est uniquement destiné
>>>>>> à
>>>>>> son ou ses destinataires. Il contient des informations confidentielles
>>>>>> qui
>>>>>> sont la propriété de 6WIND. Toute révélation, distribution ou copie des
>>>>>> informations qu'il contient est strictement interdite. Si vous avez reçu
>>>>>> ce
>>>>>> message par erreur, veuillez immédiatement le signaler à l'émetteur et
>>>>>> détruire toutes les données reçues.
>>>>>>
>>>>>> This e-mail message, including any attachments, is for the sole use of
>>>>>> the
>>>>>> intended recipient(s) and contains information that is confidential and
>>>>>> proprietary to 6WIND. All unauthorized review, use, disclosure or
>>>>>> distribution is prohibited. If you are not the intended recipient,
>>>>>> please
>>>>>> contact the sender by reply e-mail and destroy all copies of the
>>>>>> original
>>>>>> message.
>>>>>>
>>>
>>>
>>> --
>>> Ivan Boule
>>> 6WIND Development Engineer


-- 
Ivan Boule
6WIND Development Engineer