From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-qt0-f175.google.com (mail-qt0-f175.google.com [209.85.216.175]) by dpdk.org (Postfix) with ESMTP id E6F202BDF for ; Thu, 29 Mar 2018 14:08:14 +0200 (CEST) Received: by mail-qt0-f175.google.com with SMTP id s48so5929185qtb.10 for ; Thu, 29 Mar 2018 05:08:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:in-reply-to:references:from:date:message-id:subject:to; bh=D49aRboIwjfc5RPIU1Rzdx2SlbXBJrqxKQW5zDERUSE=; b=K2/sdmupyORziynMYx+LsqvmEBOIvk42ibGiwvdErZdBYKJQXxK8XgyvN1J//2b+GX 3HIaysbOgDwmr9v4BGejzWpAe5GNjw4w0Bv5I9MVOEz+mttRIjeOucHrLTRK4FOl4Tt9 oEeJtV5hOMqpj9oaHl1rvy9T09gn6MVnbTv2ElQw59aqbTp4eEIiwl8TrBC8wCwLb7Uu gHQn1GbSflZJEewmXyBaNbV0atxrW+BxqIBEByvIxS1y8flHNED7YARy+2OkFT1TcVaK 1soKh1F5NbZcKZ7MvhfBk6Xvw6ZWH1nHMXZJizS8ELL6ySV0BxTlJuM3Fg9CdIwryCl0 7zug== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to; bh=D49aRboIwjfc5RPIU1Rzdx2SlbXBJrqxKQW5zDERUSE=; b=PrQ3hVy4nb4TMPF0Zbylqca1iGk7Ov9DfLUtka/E8dXXtsOMRfCJO/yY4viBIYNmAs y8PArIYOIjG6YMR+/lWg336hM4mVNF5Ob9+w3KBVLFEdC+5r/OK5dK0ps1QYBRRj8rEg xkblzjqqNQVmHyWRDSBtI9MrS3y0/1ofEE4Ynx7mjb2dqV8xRXccJKaLKIn9DSbPtDS8 HbTcUogEqdNRc82FNosfQLpJMGkiklo7FkeyKAI6PrfAdckjNvNGPGMC/Z1Splh9uAUi 6GKnSBm0svhhRQD4Okga91TTW24CBgcX5BXrdfNAquya2m5HF3nruBNsXA5A9NBfJQ9Q aI5Q== X-Gm-Message-State: ALQs6tC2tLb5PogPrNLJEZP6dNm1pnMUAWly7acNuYkyuCG80ObhIGcK 87uuKNl2ORGn4bCze8Z/bW690qs3lQs0vhVchrmV1A== X-Google-Smtp-Source: AIpwx4+gON1lZmFJOACK9200YNkPiSFQHiNA9SgZKMCuFB68G/C9D5bup6jRJu9zEVeqS78u3pXaNe3HSDGkQJ37hX8= X-Received: by 10.200.49.99 with SMTP id h32mr10938932qtb.299.1522325293794; Thu, 29 Mar 2018 05:08:13 -0700 (PDT) MIME-Version: 1.0 Received: by 10.55.44.196 with HTTP; Thu, 29 Mar 2018 05:08:13 -0700 (PDT) In-Reply-To: References: From: Sam Date: Thu, 29 Mar 2018 20:08:13 +0800 Message-ID: To: dev@dpdk.org Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.15 Subject: Re: [dpdk-dev] Bug in "rte_eth_dev_attach" while using mlx5 pmd driver? X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 29 Mar 2018 12:08:15 -0000 I use a temporary patch to fix this bug: diff --git a/lib/librte_ether/rte_ethdev.c b/lib/librte_ether/rte_ethdev.c index 4e1e6dc..8026a05 100644 --- a/lib/librte_ether/rte_ethdev.c +++ b/lib/librte_ether/rte_ethdev.c @@ -479,7 +479,7 @@ struct rte_eth_dev * rte_eth_dev_attach(const char *devargs, uint8_t *port_id) { int ret = -1; - int current = rte_eth_dev_count(); + //int current = rte_eth_dev_count(); char *name = NULL; char *args = NULL; @@ -506,12 +506,14 @@ struct rte_eth_dev * /* if nothing happened, there is a bug here, since some driver told us * it did attach a device, but did not create a port. */ - if (current == rte_eth_dev_count()) { - ret = -1; - goto err; - } + //if (current == rte_eth_dev_count()) { + // ret = -1; + // goto err; + //} + + rte_eth_dev_get_port_by_name(name, port_id); - *port_id = eth_dev_last_created_port; + //*port_id = eth_dev_last_created_port; ret = 0; err: 2018-03-29 15:00 GMT+08:00 Sam : > Hi all, > > I'm using OVS-2.8.0 + DPDK-17.05-rc1, with 2 pf and 1 vf port under mlx5 > pmd driver, like this: > > $ sudo /usr/local/share/openvswitch/scripts/dpdk_nic_bind --status >> Network devices using DPDK-compatible driver >> ============================================ >> >> Network devices using kernel driver >> =================================== >> ... >> 0000:03:00.0 'MT27640 Family' if=eth4 drv=mlx5_core unused=igb_uio >> 0000:03:00.1 'MT27640 Family' if=eth5 drv=mlx5_core unused=igb_uio >> 0000:03:00.2 'MT27641 Family' if=enp3s0f2 drv=mlx5_core unused=igb_uio >> *Active* >> ... >> Other network devices >> ===================== >> > > > Then I start ovs-vswitchd follow these steps: > >> mkdir -p /usr/local/etc/openvswitch >> mkdir -p /usr/local/var/run/openvswitch >> [root@dpdk ~]# rm /usr/local/etc/openvswitch/conf.db >> [root@dpdk ~]# ovsdb-tool create /usr/local/etc/openvswitch/conf.db >> /usr/local/share/openvswitch/vswitch.ovsschema >> [root@dpdk ~]# ovsdb-server /usr/local/etc/openvswitch/conf.db \ >> -vconsole:emer -vsyslog:err -vfile:info \ >> --remote=punix:/usr/local/var/run/openvswitch/db.sock \ >> --private-key=db:Open_vSwitch,SSL,private_key \ >> --certificate=db:Open_vSwitch,SSL,certificate \ >> --bootstrap-ca-cert=db:Open_vSwitch,SSL,ca_cert --no-chdir \ >> --log-file=/usr/local/var/log/openvswitch/ovsdb-server.log \ >> --pidfile=/usr/local/var/run/openvswitch/ovsdb-server.pid \ >> --detach --monitor >> [root@dpdk ~]# ovs-vsctl --no-wait init >> [root@dpdk ~]# ovs-vsctl --no-wait set Open_vSwitch . >> other_config:dpdk-init=true >> [root@dpdk ~]# ovs-vsctl --no-wait set Open_vSwitch . >> other_config:dpdk-socket-mem="1024,0" >> [root@dpdk ~]# ovs-vsctl --no-wait set Open_vSwitch . >> other_config:pmd-cpu-mask=0x04 >> [root@dpdk ~]# ovs-vswitchd unix:/usr/local/var/run/openvswitch/db.sock \ >> -vconsole:emer -vsyslog:err -vfile:info --mlockall --no-chdir \ >> --log-file=/usr/local/var/log/openvswitch/ovs-vswitchd.log \ >> --pidfile=/usr/local/var/run/openvswitch/ovs-vswitchd.pid \ >> --detach --monitor > > > Then I add br0 and dpdk0 port follow these steps: > >> ovs-vsctl add-br br0 -- set bridge br0 datapath_type=netdev >> ovs-vsctl add-port br0 dpdk0 -- set Interface dpdk0 type=dpdk >> options:dpdk-devargs=0000:03:00.0 > > > After that, I got error in ovs-vswitchd.log like this: > >> 2018-03-29T02:44:11.913Z|00071|memory|INFO|handlers:22 ports:1 >> revalidators:8 rules:5 >> 2018-03-29T02:45:18.658Z|00072|netdev_dpdk|WARN|Error attaching device >> '0000:03:00.0' to DPDK >> 2018-03-29T02:45:18.658Z|00073|netdev|WARN|dpdk0: could not set >> configuration (Invalid argument) > > > Deep into code, I found this is caused by these code in rte_ethdev.c: > >> int >> rte_eth_dev_attach(const char *devargs, uint16_t *port_id) >> { >> int ret = -1; >> int current = rte_eth_dev_count(); >> ... >> /* if nothing happened, there is a bug here, since some driver told us >> * it did attach a device, but did not create a port. >> */ >> if (current == rte_eth_dev_count()) { >> ret = -1; >> goto err; >> } >> ... >> } > > > Which is the nb_ports returned by rte_eth_dev_count() keeps the same. > I think this is mlx5 driver probe port count and add into nb_ports while > starting ovs-vswitchd, not adding port by command `ovs-vsctl add port br0 > ...`. Refer to "mlx5_pci_probe" -> "rte_eth_dev_allocate" -> "eth_dev_get". > > So is this a bug or I should use some other way to add port into bridge > under mlx5 driver? Or is this bug fixed in some version? Or how to fix this > bug? Thank you~ >