From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-qt0-f174.google.com (mail-qt0-f174.google.com [209.85.216.174]) by dpdk.org (Postfix) with ESMTP id 43D222BF2 for ; Thu, 29 Mar 2018 09:00:03 +0200 (CEST) Received: by mail-qt0-f174.google.com with SMTP id g5so5251283qth.7 for ; Thu, 29 Mar 2018 00:00:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:from:date:message-id:subject:to; bh=COW52bLweXunOoZJwVICgWP9UGXMeipbDfOYoKVYL00=; b=Yp1JrWRBPdCs5JYLtFWJsN9azWw88evS0+7PKIlR8QbWTBYOF+Ndy2D9JCNxdikmCl P6HsuVo2hmIc8TaGK+Yy6dlVRjfJm+Df24D0oldAbRDyb6wIUf1JtTnY5wE1sgZaAtam AiayQUWsp/Vra7dd1BkWJj6+dTDeqqhuMIZ/yNf0Kt8SXapZdb8sQ2ttkqdPMaqPatC9 ZIrmfMDA6QeEQcWXtF3vB3y5IaY52t+KOTtuoaVbzwC5Cch8i0uoO26szYI5/i0O41wP mr/edKqaKDDAABrOtZ/jZbXlFF+kM/mSJFwIB8kWdx1b09BrUmE+SEk53Xlleke2xjtq 6wCg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:from:date:message-id:subject:to; bh=COW52bLweXunOoZJwVICgWP9UGXMeipbDfOYoKVYL00=; b=LP4QPpjzPrcJ2JD/idVNdv4qUGUpMDhTw8i6osR7YuGKhRTk3260Y1yVueIsAsrus/ 5/vGlYcIdcxFiTGvvX3ZYBwc+EXkOpJWnflVr49wK/YSLwHNpDtJC4EySxGcD8CHLggR OMCf7eQcZ6HileRNAbKEyH2t4OilVLBzsS0f//Zz0fm/qB4iEhEYqLNiuaVvLOvGXw6J oGUCnbeEO+mVDsAQAlvwBbT/bsuprQzDrzyoz5zrKls2X3QjTkwuxur5I/2sRVG6kIXj ShP5iANBOAaziH9y/IhQA1E1chVvIO7hoRq+44CtdLzNb49aqiXrNGLRf8WWE3EGIDlJ HO6Q== X-Gm-Message-State: AElRT7EeoOqy3/stwc1rqGChEg3vEoybueComFIoMEFDdxDB6qSFFgeH Y9drE0G0WkFKo6uQ3mZ977eOyMdXTSjiPMIRqdHWQw== X-Google-Smtp-Source: AIpwx49pEY2qiBFKXIeo9GtwF+hk1KRaeTb0i3WXyoTEp59Cbv4YAyR/ThkLqv9MA5BjeVyOTDxHGtDYT/GcRIPGk8s= X-Received: by 10.200.36.115 with SMTP id d48mr9517629qtd.329.1522306802430; Thu, 29 Mar 2018 00:00:02 -0700 (PDT) MIME-Version: 1.0 Received: by 10.55.44.196 with HTTP; Thu, 29 Mar 2018 00:00:01 -0700 (PDT) From: Sam Date: Thu, 29 Mar 2018 15:00:01 +0800 Message-ID: To: dev@dpdk.org Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.15 Subject: [dpdk-dev] Bug in "rte_eth_dev_attach" while using mlx5 pmd driver? X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 29 Mar 2018 07:00:04 -0000 Hi all, I'm using OVS-2.8.0 + DPDK-17.05-rc1, with 2 pf and 1 vf port under mlx5 pmd driver, like this: $ sudo /usr/local/share/openvswitch/scripts/dpdk_nic_bind --status > Network devices using DPDK-compatible driver > ============================================ > > Network devices using kernel driver > =================================== > ... > 0000:03:00.0 'MT27640 Family' if=eth4 drv=mlx5_core unused=igb_uio > 0000:03:00.1 'MT27640 Family' if=eth5 drv=mlx5_core unused=igb_uio > 0000:03:00.2 'MT27641 Family' if=enp3s0f2 drv=mlx5_core unused=igb_uio > *Active* > ... > Other network devices > ===================== > Then I start ovs-vswitchd follow these steps: > mkdir -p /usr/local/etc/openvswitch > mkdir -p /usr/local/var/run/openvswitch > [root@dpdk ~]# rm /usr/local/etc/openvswitch/conf.db > [root@dpdk ~]# ovsdb-tool create /usr/local/etc/openvswitch/conf.db > /usr/local/share/openvswitch/vswitch.ovsschema > [root@dpdk ~]# ovsdb-server /usr/local/etc/openvswitch/conf.db \ > -vconsole:emer -vsyslog:err -vfile:info \ > --remote=punix:/usr/local/var/run/openvswitch/db.sock \ > --private-key=db:Open_vSwitch,SSL,private_key \ > --certificate=db:Open_vSwitch,SSL,certificate \ > --bootstrap-ca-cert=db:Open_vSwitch,SSL,ca_cert --no-chdir \ > --log-file=/usr/local/var/log/openvswitch/ovsdb-server.log \ > --pidfile=/usr/local/var/run/openvswitch/ovsdb-server.pid \ > --detach --monitor > [root@dpdk ~]# ovs-vsctl --no-wait init > [root@dpdk ~]# ovs-vsctl --no-wait set Open_vSwitch . > other_config:dpdk-init=true > [root@dpdk ~]# ovs-vsctl --no-wait set Open_vSwitch . > other_config:dpdk-socket-mem="1024,0" > [root@dpdk ~]# ovs-vsctl --no-wait set Open_vSwitch . > other_config:pmd-cpu-mask=0x04 > [root@dpdk ~]# ovs-vswitchd unix:/usr/local/var/run/openvswitch/db.sock \ > -vconsole:emer -vsyslog:err -vfile:info --mlockall --no-chdir \ > --log-file=/usr/local/var/log/openvswitch/ovs-vswitchd.log \ > --pidfile=/usr/local/var/run/openvswitch/ovs-vswitchd.pid \ > --detach --monitor Then I add br0 and dpdk0 port follow these steps: > ovs-vsctl add-br br0 -- set bridge br0 datapath_type=netdev > ovs-vsctl add-port br0 dpdk0 -- set Interface dpdk0 type=dpdk > options:dpdk-devargs=0000:03:00.0 After that, I got error in ovs-vswitchd.log like this: > 2018-03-29T02:44:11.913Z|00071|memory|INFO|handlers:22 ports:1 > revalidators:8 rules:5 > 2018-03-29T02:45:18.658Z|00072|netdev_dpdk|WARN|Error attaching device > '0000:03:00.0' to DPDK > 2018-03-29T02:45:18.658Z|00073|netdev|WARN|dpdk0: could not set > configuration (Invalid argument) Deep into code, I found this is caused by these code in rte_ethdev.c: > int > rte_eth_dev_attach(const char *devargs, uint16_t *port_id) > { > int ret = -1; > int current = rte_eth_dev_count(); > ... > /* if nothing happened, there is a bug here, since some driver told us > * it did attach a device, but did not create a port. > */ > if (current == rte_eth_dev_count()) { > ret = -1; > goto err; > } > ... > } Which is the nb_ports returned by rte_eth_dev_count() keeps the same. I think this is mlx5 driver probe port count and add into nb_ports while starting ovs-vswitchd, not adding port by command `ovs-vsctl add port br0 ...`. Refer to "mlx5_pci_probe" -> "rte_eth_dev_allocate" -> "eth_dev_get". So is this a bug or I should use some other way to add port into bridge under mlx5 driver? Or is this bug fixed in some version? Or how to fix this bug? Thank you~