From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-wm0-f48.google.com (mail-wm0-f48.google.com [74.125.82.48]) by dpdk.org (Postfix) with ESMTP id CB9D22906 for ; Tue, 26 Apr 2016 11:12:40 +0200 (CEST) Received: by mail-wm0-f48.google.com with SMTP id n3so27408805wmn.0 for ; Tue, 26 Apr 2016 02:12:40 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=6wind-com.20150623.gappssmtp.com; s=20150623; h=subject:to:references:cc:from:message-id:date:user-agent :mime-version:in-reply-to:content-transfer-encoding; bh=tleCGjQIzW9FcXW5QbvBV95NtJff/ZJhG1FG0v5YmhI=; b=pAVo4z/cDTid1qpsFJU0hTCP0H/loWu/RPHFErRVyW3LwJs1RDFuYTXsw8RsE7f0n0 5CrUH3eebOjvPCWpXpZw6YkzXvnxNdIoUUWKM7COHIqReMLRJ+IO7gWxOBuzUCzV0Uf6 F6RXyVSsDK85337/hjkUVa60bozz1sVxttU+b0SLZ7iYvg6nQ/0jc/vQnIcX7FmwZGH1 9whb7JvTvNy2i+OwkQnAUm9D6Ls063lL0T5JdvmsfKd6Yct0ODgexq7+uJC/y6Y/8rWn hvBlXnoxeLbWzJkbv8MVb4PNAsxcJ/5xR46libuIGWNCYQ+eltbYok5uUv8tR3+OSHK2 02ew== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:subject:to:references:cc:from:message-id:date :user-agent:mime-version:in-reply-to:content-transfer-encoding; bh=tleCGjQIzW9FcXW5QbvBV95NtJff/ZJhG1FG0v5YmhI=; b=kr62oNPjIHlNX/LRlRFSdA9Qcy+2ahXyx0o/Z+7lE4b5uqj+Kjub6+uM1eF9+hEz9n bHV0iIkGEhHJ1FICo9YL8WvFoUU8CzO8NZU3A1dZYaZU10WdKfPWNg8vrGqLzNPe7/Gh zTSj5+nagQPQ8mUIUwSofNA01fc98d98HvGRcdYZqb9IMib8O7dAA1VaCYY70Rih4ZN8 QgEGYjwNLNZ2S6O4RM5uFrBI0ylcR0gdJlXCAxFg9p3DN2kdDr4xYNumGZZthnBcR1xA lLo7cjJniJy5ep1nlbSVPzEpIFaULxPSPClLb+1cnjveX/aJMfJ6N6oXE1B8J72Am5/8 B23A== X-Gm-Message-State: AOPr4FWnbLhOU5u9ayORAffIaSWdQKNeG96lt122Myo9hxn3E28Z/b3ShmYY9mAjrPCe+ybS X-Received: by 10.194.105.230 with SMTP id gp6mr1766105wjb.90.1461661960559; Tue, 26 Apr 2016 02:12:40 -0700 (PDT) Received: from [10.16.0.189] (guy78-3-82-239-227-177.fbx.proxad.net. [82.239.227.177]) by smtp.gmail.com with ESMTPSA id m140sm21032944wma.24.2016.04.26.02.12.39 (version=TLSv1/SSLv3 cipher=OTHER); Tue, 26 Apr 2016 02:12:39 -0700 (PDT) To: santosh References: <5719D0DD.6070706@6wind.com> Cc: "dev@dpdk.org" From: Ivan Boule Message-ID: <571F30FC.8060203@6wind.com> Date: Tue, 26 Apr 2016 11:12:28 +0200 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:38.0) Gecko/20100101 Icedove/38.7.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8bit Subject: Re: [dpdk-dev] ixgbe : query regarding your code changes for VF mac add X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: patches and discussions about DPDK List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 26 Apr 2016 09:12:41 -0000 Hi Santosh, Things are a little bit more complex. When the permanent MAC address of a VF is assigned by the PF driver, it cannot be changed nor deleted by the VF through commands issued to the PF driver. In this case, the function xgbevf_remove_mac_addr() must not add the permanent MAC address again, as currently done. Conversely, when the permanent MAC address of a VF is assigned by the Vf itself, then it will be also deleted by the call to ixgbevf_set_uc_addr_vf(hw, 0, NULL) in the function xgbevf_remove_mac_addr (as other MAC addresses that have been previously added through a call to the function ixgbevf_add_mac_addr). In this case, the function xgbevf_remove_mac_addr must effectively add the permanent MAC address again. To resume, the ixgbevf PMD must record in a new dedicated flag if the permanent MAC address of a VF has been assigned by the PF driver or not, and test this flag in the function xgbevf_remove_mac_addr to decide if it must add again the permanent MAC address of the VF. Regards, Ivan On 04/26/2016 07:36 AM, santosh wrote: > Hi > > Looks like there is a bug in API "ixgbevf_remove_mac_addr()". For > deleting a given MAC it deletes all MAC (including permanent MAC) and > while adding (it does after few statements), it skips to add permanent > MAC. > > ixgbevf_remove_mac_addr(struct rte_eth_dev *dev, uint32_t index) > { > ....... > > /* > * The IXGBE_VF_SET_MACVLAN command of the ixgbe-pf driver does > * not support the deletion of a given MAC address. > * Instead, it imposes to delete all MAC addresses, then to add again > * all MAC addresses with the exception of the one to be deleted. > */ > > > (void) ixgbevf_set_uc_addr_vf(hw, 0, NULL); > ... > > 2. Following modification at "ixgbevf_remove_mac_addr()} also > helped to make ping works > > $ git diff dpdk/lib/librte_pmd_ixgbe/ixgbe_ethdev.c > --- a/dpdk/lib/librte_pmd_ixgbe/ixgbe_ethdev.c > +++ b/dpdk/lib/librte_pmd_ixgbe/ixgbe_ethdev.c > > @@ -3530,10 +3557,7 @@ ixgbevf_remove_mac_addr(struct > rte_eth_dev *dev, uint32_t index) > /* Skip NULL MAC addresses */ > if (is_zero_ether_addr(mac_addr)) > continue; > - /* Skip the permanent MAC address */ > - if (memcmp(perm_addr, mac_addr, sizeof(struct ether_addr)) == 0) > - continue; > > On Mon, Apr 25, 2016 at 7:05 PM, santosh wrote: >> Hi Ivan >> >> ixgbevf_set_default_mac_addr() could not find in our code base. >> put traces at other places as suggested by you. >> Log at "eth_ixgbevf_dev_init" never displayed >> Rest logs displayed as shown below. I re-build the driver module and >> loaded on our virtual router and rebooted the system. >> >> 1. >> Logs at the time of boot up: >> ----------------------------------------- >> >> INIT: Initializing NIC port 0 RX queue 0 ... >> INIT: Initializing NIC port 0 TX queue 0 ... >> Santosh ixgbevf_add_mac_addr portid=0 mac=00:50:56:A0:10:C2 >> ..... >> .... >> >> Santosh ixgbevf_add_mac_addr portid=0 mac=00:50:56:A0:10:C2 >> RUNTIME: Detected port 0 status changed to UP. >> >> 2. >> a. >> Configured a new MAC at Virtual Router CLI >> >> root@mx86-bgl-2-r1# set interfaces ge-0/0/0 mac 00:50:56:a0:a0:c3 >> >> [edit] >> root@mx86-bgl-2-r1# commit >> commit complete >> >> [edit] >> root@mx86-bgl-2-r1# show interfaces >> ge-0/0/0 { >> mac 00:50:56:a0:a0:c3; >> unit 0 { >> family inet { >> address 10.1.1.102/24; >> } >> } >> } >> >> [edit] >> root@mx86-bgl-2-r1# run ping 10.1.1.101 >> PING 10.1.1.101 (10.1.1.101): 56 data bytes >> 64 bytes from 10.1.1.101: icmp_seq=0 ttl=64 time=2.142 ms >> 64 bytes from 10.1.1.101: icmp_seq=1 ttl=64 time=8.465 ms >> ^C >> --- 10.1.1.101 ping statistics --- >> 2 packets transmitted, 2 packets received, 0% packet loss >> round-trip min/avg/max/stddev = 2.142/5.303/8.465/3.162 ms >> >> [edit] >> root@mx86-bgl-2-r1# >> >> b. >> Logs at console for above config: >> ------------------------------------------------- >> >> root@localhost:/home/pfe# Santosh ixgbevf_add_mac_addr portid=0 >> mac=00:50:56:A0:A0:C3 >> >> root@localhost:/home/pfe# >> >> >> 3. >> a. Deleted above config MAC >> root@mx86-bgl-2-r1# delete interfaces ge-0/0/0 mac >> >> [edit] >> root@mx86-bgl-2-r1# commit >> commit complete >> >> [edit] >> root@mx86-bgl-2-r1# show interfaces >> ge-0/0/0 { >> unit 0 { >> family inet { >> address 10.1.1.102/24; >> } >> } >> } >> >> [edit] >> root@mx86-bgl-2-r1# run ping 10.1.1.101 >> PING 10.1.1.101 (10.1.1.101): 56 data bytes >> ^C >> --- 10.1.1.101 ping statistics --- >> 3 packets transmitted, 0 packets received, 100% packet loss >> >> [edit] >> root@mx86-bgl-2-r1# >> >> b. >> Logs at console for above cofig >> ------------------------- >> >> >> root@localhost:/home/pfe# ixgbevf_remove_mac_addr santosh portid=0 >> index=1 mac=00:50:56:A0:A0:C3 >> >> >> >> Thanks >> Santosh >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> On Fri, Apr 22, 2016 at 12:51 PM, Ivan Boule wrote: >>> Hi Santosh, >>> >>> My job at 6WIND does not consist in answering to DPDK questions. In general, >>> I have other priorities, including vacations... >>> In the meantime, nobody prevents you to add traces in the code to really >>> understand what happens, as suggested in my last answer. >>> >>> Regards, >>> Ivan >>> >>> >>> On 04/21/2016 07:55 AM, santosh wrote: >>>> >>>> Hi Ivan and team, >>>> >>>> Please respond to my last mail and let me know if there is any >>>> alternate way to handle this. >>>> Our release is in pending due to this issue. >>>> >>>> >>>> Thanks & Regards >>>> Santosh >>>> >>>> On Wed, Apr 20, 2016 at 2:35 PM, santosh wrote: >>>>> >>>>> Hi Ivan, >>>>> >>>>> Thanks for your response. >>>>> >>>>> Let me explain you the issue that we are facing on our virtual router >>>>> in VMware environment. >>>>> >>>>> 1. We are using ixgbe driver , SRIOV enabled . >>>>> root@localhost:~# lspci >>>>> .... "Intel Corporation 82599 Ethernet Controller Virtual >>>>> Function" >>>>> >>>>> 2. "mx86-bgl-1-r1" is our router under testing and R2 is a standard >>>>> router. >>>>> >>>>> mx86-bgl-1-r1 is connected to R2 >>>>> mx86-bgl-1-r1 (10.1.1.103)------------------>R2(10.1.1.101) >>>>> >>>>> 2. This time ping test passes >>>>> >>>>> root@mx86-bgl-1-r1# show interfaces >>>>> ge-0/0/0 { >>>>> unit 0 { >>>>> family inet { >>>>> address 10.1.1.103/24; >>>>> } >>>>> } >>>>> } >>>>> >>>>> [edit] >>>>> root@mx86-bgl-1-r1# >>>>> root@mx86-bgl-1-r1# run ping 10.1.1.101 >>>>> PING 10.1.1.101 (10.1.1.101): 56 data bytes >>>>> 64 bytes from 10.1.1.101: icmp_seq=0 ttl=64 time=0.980 ms >>>>> 64 bytes from 10.1.1.101: icmp_seq=1 ttl=64 time=1.433 ms >>>>> >>>>> >>>>> 3. Default MAC address of interface ge-0/0/0 is as shown below: >>>>> >>>>> root@mx86-bgl-1-r1# run show interfaces ge-0/0/0 extensive | grep current >>>>> Current address: 02:06:0a:0e:ff:f0, Hardware address: >>>>> 02:06:0a:0e:ff:f0 >>>>> >>>>> [edit] >>>>> root@mx86-bgl-1-r1# >>>>> >>>>> >>>>> 4. Now I am changing MAC. Ping works this time also >>>>> >>>>> root@mx86-bgl-1-r1# set interfaces ge-0/0/0 mac 02:06:0a:0a:ff:f0 >>>>> >>>>> [edit] >>>>> root@mx86-bgl-1-r1# commit >>>>> commit complete >>>>> >>>>> [edit] >>>>> root@mx86-bgl-1-r1# run show interfaces ge-0/0/0 extensive | grep current >>>>> Current address: 02:06:0a:0a:ff:f0, Hardware address: >>>>> 02:06:0a:0e:ff:f0 >>>>> >>>>> [edit] >>>>> root@mx86-bgl-1-r1# show interfaces >>>>> ge-0/0/0 { >>>>> mac 02:06:0a:0a:ff:f0; >>>>> unit 0 { >>>>> family inet { >>>>> address 10.1.1.103/24; >>>>> } >>>>> } >>>>> } >>>>> >>>>> [edit] >>>>> root@mx86-bgl-1-r1# >>>>> root@mx86-bgl-1-r1# run ping 10.1.1.101 >>>>> PING 10.1.1.101 (10.1.1.101): 56 data bytes >>>>> 64 bytes from 10.1.1.101: icmp_seq=0 ttl=64 time=1.338 ms >>>>> 64 bytes from 10.1.1.101: icmp_seq=1 ttl=64 time=8.832 ms >>>>> 64 bytes from 10.1.1.101: icmp_seq=2 ttl=64 time=1.292 ms >>>>> >>>>> >>>>> 5. I am deleting the above configured MAC. >>>>> In this case newly configured MAC as shown above will be deleted >>>>> and Default MAC (02:06:0a:0e:ff:f0) >>>>> will be applied. Ping fails at this step. "return statement >>>>> added by you is not allowing to configure default MAC. >>>>> >>>>> >>>>> [edit] >>>>> root@mx86-bgl-1-r1# delete interfaces ge-0/0/0 mac >>>>> >>>>> [edit] >>>>> root@mx86-bgl-1-r1# commit >>>>> commit complete >>>>> >>>>> [edit] >>>>> root@mx86-bgl-1-r1# show interfaces >>>>> ge-0/0/0 { >>>>> unit 0 { >>>>> family inet { >>>>> address 10.1.1.103/24; >>>>> } >>>>> } >>>>> } >>>>> >>>>> [edit] >>>>> >>>>> root@mx86-bgl-1-r1# run show interfaces ge-0/0/0 extensive | grep >>>>> current // Displays value from router's local database not from >>>>> NIC. >>>>> Current address: 02:06:0a:0e:ff:f0, Hardware address: >>>>> 02:06:0a:0e:ff:f0 >>>>> >>>>> [edit] >>>>> root@mx86-bgl-1-r1# run ping 10.1.1.101 >>>>> PING 10.1.1.101 (10.1.1.101): 56 data bytes >>>>> ^C >>>>> 2 packets transmitted, 0 packets received, 100% packet loss >>>>> >>>>> [edit] >>>>> root@mx86-bgl-1-r1# >>>>> >>>>> >>>>> 7. LOGs: (I have added a debug log just before the return statement >>>>> that you place) >>>>> >>>>> Log o/p in failure case >>>>> .... >>>>> Santosh ixgbevf_add_mac_addr returning >>>>> .... >>>>> >>>>> code changes: >>>>> ----------------------------- >>>>> ixgbevf_add_mac_addr(....) { >>>>> ... >>>>> if (memcmp(hw->mac.perm_addr, mac_addr, sizeof(struct ether_addr)) >>>>> == 0) { >>>>> PMD_DRV_LOG(DEBUG, "Existing MAC \n"); >>>>> printf("Santosh %s returning \n",__FUNCTION__); >>>>> return; >>>>> } >>>>> >>>>> >>>>> 8. If I remove the above "return" statement, build the driver, loaded >>>>> in router and repeat the steps-2 to steps-5 of MAC add and MAC delete >>>>> on our router then ping passes. >>>>> >>>>> root@mx86-bgl-1-r1# run ping 10.1.1.101 >>>>> PING 10.1.1.101 (10.1.1.101): 56 data bytes >>>>> 64 bytes from 10.1.1.101: icmp_seq=0 ttl=64 time=2.356 ms >>>>> 64 bytes from 10.1.1.101: icmp_seq=1 ttl=64 time=1.415 ms >>>>> 64 bytes from 10.1.1.101: icmp_seq=2 ttl=64 time=1.226 ms >>>>> ^C >>>>> --- 10.1.1.101 ping statistics --- >>>>> 3 packets transmitted, 3 packets received, 0% packet loss >>>>> round-trip min/avg/max/stddev = 1.226/1.666/2.356/0.494 ms >>>>> >>>>> >>>>> 9. Please let me know whether is it fine to remove that return >>>>> statement added by you in "ixgbevf_add_mac_addr" ? >>>>> >>>>> >>>>> >>>>> Thanks & Regards, >>>>> Santosh >>>>> >>>>> On Wed, Apr 20, 2016 at 3:31 AM, Ivan Boule wrote: >>>>>> >>>>>> Hi Santosh, >>>>>> >>>>>> I do not get exactly what you attempt to do on a VF. >>>>>> Are you first deleting the so-called permanent MAC address by a call to >>>>>> the >>>>>> function ixgbevf_remove_mac_addr() ? This operation is not allowed. >>>>>> Can you explain exactly the sequence of operations that are done, so >>>>>> that I >>>>>> can understand how the test (memcmp(hw->mac.perm_addr, mac_addr, >>>>>> sizeof(struct ether_addr)) == 0) in the function ixgbevf_add_mac_addr() >>>>>> prevents them to be successfully performed. >>>>>> >>>>>> Ivan >>>>>> >>>>>> PS : please, can you CC your emails to dev@dpdk.org >>>>>> >>>>>> >>>>>> 2016-04-19 17:01 GMT+02:00 santosh : >>>>>>> >>>>>>> >>>>>>> Hi Ivan, >>>>>>> >>>>>>> Your following code changes causing issue in Vmware environment. >>>>>>> >>>>>>> ----------------------------------- ------------------- >>>>>>> ------------------------------ >>>>>>> + /* >>>>>>> + * On a 82599 VF, adding again the same MAC addr is not an idempotent >>>>>>> + * operation. Trap this case to avoid exhausting the [very limited] >>>>>>> + * set of PF resources used to store VF MAC addresses. >>>>>>> + */ >>>>>>> + if (memcmp(hw->mac.perm_addr, mac_addr, sizeof(struct ether_addr)) == >>>>>>> 0) >>>>>>> + return; >>>>>>> diag = ixgbevf_set_uc_addr_vf(hw, 2, mac_addr->addr_bytes); >>>>>>> -------------------------------------------------------------------- >>>>>>> ------------- ------- >>>>>>> >>>>>>> >>>>>>> Issue: >>>>>>> At CLI , we have provision to add /del MAC of an interface. >>>>>>> During MAC delete, existing MAC is deleted and default MAC is applied. >>>>>>> This default MAC is not being applied in VMware environment >>>>>>> successfully due to "return" statement >>>>>>> in your above code changes. As a result traffic is stopped completely. >>>>>>> If I remove above >>>>>>> "return" statement then traffic continues to flow after MAC delete. >>>>>>> >>>>>>> Please let me know your suggestion to handle this scenario . If I >>>>>>> remove "return" what will be the consequences ? >>>>>>> >>>>>>> If removing "return" statement is not good idea then what are other >>>>>>> way to handle MAC delete scenario ? we have only 1 VF per PF in our >>>>>>> setup as of now. >>>>>>> >>>>>>> >>>>>>> Thanks >>>>>>> Santosh >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> Ivan BOULE >>>>>> 6WIND >>>>>> Software Engineer >>>>>> >>>>>> Tel: +33 1 39 30 92 47 >>>>>> Mob: + 33 6 77 25 26 38 >>>>>> Fax: +33 1 39 30 92 11 >>>>>> ivan.boule@6wind.com >>>>>> www.6wind.com >>>>>> Join the Multicore Packet Processing Forum: >>>>>> www.multicorepacketprocessing.com >>>>>> >>>>>> Ce courriel ainsi que toutes les pièces jointes, est uniquement destiné >>>>>> à >>>>>> son ou ses destinataires. Il contient des informations confidentielles >>>>>> qui >>>>>> sont la propriété de 6WIND. Toute révélation, distribution ou copie des >>>>>> informations qu'il contient est strictement interdite. Si vous avez reçu >>>>>> ce >>>>>> message par erreur, veuillez immédiatement le signaler à l'émetteur et >>>>>> détruire toutes les données reçues. >>>>>> >>>>>> This e-mail message, including any attachments, is for the sole use of >>>>>> the >>>>>> intended recipient(s) and contains information that is confidential and >>>>>> proprietary to 6WIND. All unauthorized review, use, disclosure or >>>>>> distribution is prohibited. If you are not the intended recipient, >>>>>> please >>>>>> contact the sender by reply e-mail and destroy all copies of the >>>>>> original >>>>>> message. >>>>>> >>> >>> >>> -- >>> Ivan Boule >>> 6WIND Development Engineer -- Ivan Boule 6WIND Development Engineer