From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-qt0-f179.google.com (mail-qt0-f179.google.com [209.85.216.179]) by dpdk.org (Postfix) with ESMTP id 197A91B36E for ; Tue, 30 Jan 2018 03:23:25 +0100 (CET) Received: by mail-qt0-f179.google.com with SMTP id s39so15402852qth.7 for ; Mon, 29 Jan 2018 18:23:25 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:in-reply-to:references:from:date:message-id:subject:to; bh=3z0p2Szm9xWPNnxRlWVEmFpObLu4cJIhJZsaH1g/+NM=; b=qdhW0PMLvhl/qlqx6bC7ArVjXHbJ+w+rtKTafDyAKN0heb790BIrxdT83jdj42Gj2g tDFzFe9L/xXvj/24qm5Xx6q+sazVoKZW1698vXIFOHoDoCAVcXEcien9tK3gqlH5vGgb 5uhvhfvlFizhXwxu8hiff9VK1lsy/eqZmV9exTIbzbceIVj1m451qFvwXtkU1HokiCrA 3mSyoVC0q5ZKqocLMfj+rLxlfPfuC8aIQ2aMWgr19KkF+7I4lNl0rgS6oLVHnWeMtkjb /hvRsbACc9Ezh8YvGM10z+W1dns42GOQ736x1VYmlSz4fGwyoCihd+rHWuINyTY4LVMQ en3Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to; bh=3z0p2Szm9xWPNnxRlWVEmFpObLu4cJIhJZsaH1g/+NM=; b=XnsM4H3NQbddjUmVMZ9IMC9oV6H9TpjdI+r3a9lQsQUxM3RRGRDuDib6ab+IRRphMi eUongSoZPdkE+baGWffm7KUy+Ye19jHHX6oTBhH+677lgfI4DFnzsdT2i/09xoSx0CsC X+1RVTlMMxJqX1EXcGi1KC5MtOoBKuG8SHxuR2+wdzOUWRhJYkZrTro4lVo24dpvJkKX qkgGe2vkIcy3s+80uFkO+N3+cSbN9PZXvR+VO7kZhFjtlnFzBlZg9+F7GPGMujfODqwR Bw4BZhZjb2xlrcRejhX4o/Rr9oAmWEhVpOYexGs8Ee7CHrI0kUTLoSIogrSet/2YdmBc UmOQ== X-Gm-Message-State: AKwxytdgnf4XcNRSR8Qc5J/szV8ySc/r4S+N0o349apVKnxQQ8MibVuY OhxdImyaNf5G5jGpRiU7b+59P0JjMHOx2ZrTNztWVQ== X-Google-Smtp-Source: AH8x224r3f73GLVtW9GYWPcSDNLMD2G6DyZ/u5Rmm6LucWd95B2bz7vkVKLij9HkWrgusyJW2xD95lkgVCJrop3X88M= X-Received: by 10.200.43.193 with SMTP id n1mr41880937qtn.20.1517279004389; Mon, 29 Jan 2018 18:23:24 -0800 (PST) MIME-Version: 1.0 Received: by 10.140.92.179 with HTTP; Mon, 29 Jan 2018 18:23:23 -0800 (PST) In-Reply-To: References: From: Sam Date: Tue, 30 Jan 2018 10:23:23 +0800 Message-ID: To: dev@dpdk.org, ovs-dev@openvswitch.org Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.15 Subject: Re: [dpdk-dev] kernel crash bug caused by ixgbevf kernel module of centos-3.10.0-229.20.1.el7 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 30 Jan 2018 02:23:25 -0000 detail as below, bug is happened on bond of enp1s16 and enp1s16f1 [huanghuai-test@yf-mos-test-net14 ~]$ sudo /usr/local/share/openvswitch/scripts/dpdk_nic_bind --status Network devices using DPDK-compatible driver =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D 0000:01:00.0 'Ethernet Controller 10-Gigabit X540-AT2' drv=3Digb_uio unused=3Dixgbe 0000:01:00.1 'Ethernet Controller 10-Gigabit X540-AT2' drv=3Digb_uio unused=3Dixgbe Network devices using kernel driver =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D 0000:01:10.0 'X540 Ethernet Controller Virtual Function' if=3Denp1s16 drv=3Dixgbevf unused=3Dbak,igb_uio 0000:01:10.1 'X540 Ethernet Controller Virtual Function' if=3Denp1s16f1 drv=3Dixgbevf unused=3Dbak,igb_uio 0000:08:00.0 'I350 Gigabit Network Connection' if=3Deth2 drv=3Digb unused=3Digb_uio 0000:08:00.1 'I350 Gigabit Network Connection' if=3Deth3 drv=3Digb unused=3Digb_uio Other network devices =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D 2018-01-30 10:19 GMT+08:00 Sam : > I found a bug about ixgbevf kernel module in centos-3.10.0-229.20.1.el7. > And this bug is also in 3.10.0-514.10.2.el7. > > How to produce this bug: use SRIOV first, then add lots of network traffi= c > on vf port, and then ifdow/ifup vf port, after many times, this bug happe= ns. > > BUG: > > [308026.586026] ixgbevf 0000:01:10.0: NIC Link is Down > [308026.586037] ixgbevf 0000:01:10.1: NIC Link is Down > [308026.683724] bonding: bond1: link status definitely down for interface= enp1s16, disabling it > [308026.683728] bonding: bond1: now running without any active interface = ! > [308026.683729] bonding: bond1: link status definitely down for interface= enp1s16f1, disabling it > [308028.266060] bonding: bond1: Removing slave enp1s16. > [308028.266135] bonding: bond1: Warning: the permanent HWaddr of enp1s16 = - 4e:cd:a6:59:26:2c - is still in use by bond1. Set the HWaddr of enp1s16 t= o a different address to avoid conflicts. > [308028.266139] bonding: bond1: releasing active interface enp1s16 > [308028.359872] BUG: unable to handle kernel NULL pointer dereference at = 0000000000000008 > [308028.361319] IP: [] ixgbevf_alloc_rx_buffers+0x60/0x= 160 [ixgbevf] > [308028.362049] PGD 0 > [308028.362777] Oops: 0000 [#1] SMP > [308028.363481] Modules linked in: ixgbevf(OF) igb_uio(OF) iptable_mangle= iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntra= ck iptable_filter nbd(OF) vhost_net macvtap macvlan udp_diag unix_diag af_p= acket_diag netlink_diag tun tcp_diag inet_diag uio bonding ext4 mbcache jbd= 2 intel_powerclamp coretemp kvm_intel kvm crct10dif_pclmul crc32_pclmul crc= 32c_intel ghash_clmulni_intel mgag200 aesni_intel iTCO_wdt lrw dcdbas gf128= mul syscopyarea sysfillrect iTCO_vendor_support glue_helper sysimgblt ablk_= helper ttm cryptd ipmi_devintf igb ixgbe drm_kms_helper drm i2c_algo_bit pt= p i2c_core ipmi_si pps_core sg mdio ipmi_msghandler dca sb_edac mei_me mei = shpchp lpc_ich pcspkr mfd_core edac_core wmi acpi_power_meter acpi_pad ip_t= ables xfs libcrc32c sd_mod crc_t10dif crct10dif_common ahci libahci > [308028.368487] libata megaraid_sas [last unloaded: ixgbevf] > [308028.369345] CPU: 0 PID: 21971 Comm: kworker/0:1 Tainted: GF W = O-------------- 3.10.0-229.el7.x86_64 #1 > [308028.370226] Hardware name: Dell Inc. PowerEdge R720/068CDY, BIOS 2.5.= 2 01/28/2015 > [308028.371132] Workqueue: events ixgbevf_service_task [ixgbevf] > [308028.372038] task: ffff88022b0dad80 ti: ffff88010905c000 task.ti: ffff= 88010905c000 > [308028.372965] RIP: 0010:[] [] ixgb= evf_alloc_rx_buffers+0x60/0x160 [ixgbevf] > [308028.373949] RSP: 0018:ffff88010905fd10 EFLAGS: 00010287 > [308028.374900] RAX: 0000000000000200 RBX: 0000000000000000 RCX: 00000000= 00000000 > [308028.375895] RDX: 0000000000000000 RSI: 00000000000001ff RDI: ffff8800= b82061c0 > [308028.376841] RBP: ffff88010905fd48 R08: 0000000000000282 R09: 00000000= 00000001 > [308028.377780] R10: 0000000000000004 R11: 0000000000000005 R12: 00000000= 00000000 > [308028.378702] R13: 00000000fffffe00 R14: 00000000000001ff R15: ffff8800= b82061c0 > [308028.379628] FS: 0000000000000000(0000) GS:ffff882f7fa00000(0000) knl= GS:0000000000000000 > [308028.380540] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 > [308028.381471] CR2: 0000000000000008 CR3: 000000000190a000 CR4: 00000000= 001427f0 > [308028.382376] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 00000000= 00000000 > [308028.383291] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 00000000= 00000400 > [308028.384180] Stack: > [308028.385051] ffff8832d1b58bc0 ffff88010905fd28 ffff8832d1b588c0 00000= 00000000009 > [308028.385933] ffff8832d1b58bc0 ffff8800b82061c0 0000000000001028 ffff8= 8010905fdb8 > [308028.386804] ffffffffa0496ba3 ffff8832d1b58e58 000000022b1e2000 00000= 000819e2108 > [308028.387693] Call Trace: > [308028.388520] [] ixgbevf_configure+0x5d3/0x7d0 [ixgb= evf] > [308028.389363] [] ixgbevf_reinit_locked+0x65/0x90 [ix= gbevf] > [308028.390213] [] ixgbevf_service_task+0x324/0x420 [i= xgbevf] > [308028.391043] [] process_one_work+0x17b/0x470 > [308028.391888] [] worker_thread+0x11b/0x400 > [308028.392728] [] ? rescuer_thread+0x400/0x400 > [308028.393576] [] kthread+0xcf/0xe0 > [308028.394434] [] ? kthread_create_on_node+0x140/0x14= 0 > [308028.395339] [] ret_from_fork+0x7c/0xb0 > [308028.396205] [] ? kthread_create_on_node+0x140/0x14= 0 > [308028.397068] Code: c5 41 89 f6 49 89 c4 48 8d 14 40 48 8b 47 28 49 c1 = e4 04 4c 03 67 20 48 8d 1c d0 0f b7 47 4c 41 29 c5 66 0f 1f 84 00 00 00 00 = 00 <48> 83 7b 08 00 74 73 8b 53 10 48 8b 03 48 01 d0 49 83 c4 10 48 > [308028.398959] RIP [] ixgbevf_alloc_rx_buffers+0x60/0= x160 [ixgbevf] > [308028.399910] RSP > [308028.400846] CR2: 0000000000000008 > > >