BUG: soft lockup - CPU#2 stuck for 22s!

whpu000625

我有一套云环境，部署的是openstack，其中有一台机器出现了以下错误?
请问是什么原因导致的，如何解决

whpu000625

1. Mar 13 16:39:17 controller kernel: BUG: soft lockup - CPU#14 stuck for 23s! [ip:94135]
   
2. Mar 13 16:39:17 controller kernel: Modules linked in: ip_set_hash_net ip_set nf_conntrack_netlink nfnetlink xt_REDIRECT nf_conntrack_ipv6 nf_defrag_ipv6 ip6table_mangle ip6table_raw ip6table_filter ip6_tables xt_nat xt_mark xt_connmark xt_comment xt_CHECKSUM iptable_mangle binfmt_misc iptable_raw openvswitch bridge stp llc dm_thin_pool dm_persistent_data dm_bio_prison dm_bufio loop ipt_REJECT xt_conntrack iptable_filter ipt_MASQUERADE nf_nat_masquerade_ipv4 xt_addrtype iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack intel_powerclamp coretemp intel_rapl kvm_intel kvm crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul iTCO_wdt iTCO_vendor_support glue_helper ablk_helper cryptd lpc_ich mfd_core sg pcspkr mei_me mei mxm_wmi ipmi_devintf sb_edac i2c_i801 edac_core ipmi_si ipmi_msghandler
   
3. Mar 13 16:39:17 controller kernel: acpi_pad shpchp wmi ioatdma acpi_power_meter tpm_crb nfsd auth_rpcgss nfs_acl lockd grace sunrpc ip_tables xfs libcrc32c sd_mod crc_t10dif crct10dif_generic mgag200 syscopyarea sysfillrect sysimgblt drm_kms_helper crct10dif_pclmul crct10dif_common ttm crc32c_intel drm ahci igb libahci ptp libata pps_core dca megaraid_sas i2c_algo_bit i2c_core dm_mirror dm_region_hash dm_log dm_mod
   
4. Mar 13 16:39:17 controller kernel: CPU: 14 PID: 94135 Comm: ip Tainted: G             L ------------   3.10.0-327.ctrust.el7.x86_64 #1
   
5. Mar 13 16:39:17 controller kernel: Hardware name: Intel Corporation S2600CWR/S2600CWR, BIOS SE5C610.86B.11.01.0132.051220160603 05/12/2016
   
6. Mar 13 16:39:17 controller kernel: task: ffff880827ec1700 ti: ffff8801137a0000 task.ti: ffff8801137a0000
   
7. Mar 13 16:39:17 controller kernel: RIP: 0010:[<ffffffff810ae747>]  [<ffffffff810ae747>] lg_local_lock+0x47/0x60
   
8. Mar 13 16:39:17 controller kernel: RSP: 0018:ffff8801137a3dc0  EFLAGS: 00000202
   
9. Mar 13 16:39:17 controller kernel: RAX: 0000000000007fba RBX: 0000000ba22f2120 RCX: 0000000000008c8e
   
10. Mar 13 16:39:17 controller kernel: RDX: 0000000000008c90 RSI: 0000000000008c90 RDI: ffff88105eacfd30
    
11. Mar 13 16:39:17 controller kernel: RBP: ffff8801137a3dc0 R08: ffffffff81a6e870 R09: 00000000000021be
    
12. Mar 13 16:39:17 controller kernel: R10: ffffffff811fe549 R11: ffffea000688bc00 R12: ffff8801137a3d70
    
13. Mar 13 16:39:17 controller kernel: R13: 000000000000e698 R14: 00000000000039a6 R15: 000000018040002b
    
14. Mar 13 16:39:17 controller kernel: FS:  00007f75cb7e7740(0000) GS:ffff88105eac0000(0000) knlGS:0000000000000000
    
15. Mar 13 16:39:17 controller kernel: CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
    
16. Mar 13 16:39:17 controller kernel: CR2: 00000000061d68f0 CR3: 000000000194e000 CR4: 00000000001407e0
    
17. Mar 13 16:39:17 controller kernel: DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
    
18. Mar 13 16:39:17 controller kernel: DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
    
19. Mar 13 16:39:17 controller kernel: Stack:
    
20. Mar 13 16:39:17 controller kernel: ffff8801137a3de8 ffffffff811fed01 ffff8801137a3e08 ffff8801a22f2000
    
21. Mar 13 16:39:17 controller kernel: ffff8801c4ec1100 ffff8801137a3df8 ffffffff811fee24 ffff8801137a3e40
    
22. Mar 13 16:39:17 controller kernel: ffffffff811fef4e ffff8801c4ec3000 ffff88013ea4c200 000000009bc3ecf7
    
23. Mar 13 16:39:17 controller kernel: Call Trace:
    
24. Mar 13 16:39:17 controller kernel: [<ffffffff811fed01>] mntput_no_expire+0x21/0x120
    
25. Mar 13 16:39:17 controller kernel: [<ffffffff811fee24>] mntput+0x24/0x40
    
26. Mar 13 16:39:17 controller kernel: [<ffffffff811fef4e>] namespace_unlock+0x10e/0x130
    
27. Mar 13 16:39:17 controller kernel: [<ffffffff812017db>] put_mnt_ns+0x4b/0x60
    
28. Mar 13 16:39:17 controller kernel: [<ffffffff810ab0bb>] free_nsproxy+0x1b/0x90
    
29. Mar 13 16:39:17 controller kernel: [<ffffffff810ab305>] switch_task_namespaces+0x45/0x50
    
30. Mar 13 16:39:17 controller kernel: [<ffffffff810ab320>] exit_task_namespaces+0x10/0x20
    
31. Mar 13 16:39:17 controller kernel: [<ffffffff810815e6>] do_exit+0x2c6/0xa60
    
32. Mar 13 16:39:17 controller kernel: [<ffffffff816431c9>] ? __do_page_fault+0x1f9/0x420
    
33. Mar 13 16:39:17 controller kernel: [<ffffffff81081dff>] do_group_exit+0x3f/0xa0
    
34. Mar 13 16:39:17 controller kernel: [<ffffffff81081e74>] SyS_exit_group+0x14/0x20
    
35. Mar 13 16:39:17 controller kernel: [<ffffffff81647c09>] system_call_fastpath+0x16/0x1b
    
36. Mar 13 16:39:17 controller kernel: Code: c1 07 89 c2 c1 ea 10 66 39 c2 75 02 5d c3 83 e2 fe 0f b7 f2 b8 00 80 00 00 eb 10 66 0f 1f 84 00 00 00 00 00 f3 90 83 e8 01 74 0a <0f> b7 0f 66 39 ca 75 f1 5d c3 0f 1f 80 00 00 00 00 eb d6 66 0f

复制代码