Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

大佬帮忙看下是不是swap导致的内核崩溃 #641

Open
yuan6975 opened this issue Oct 5, 2024 · 0 comments
Open

大佬帮忙看下是不是swap导致的内核崩溃 #641

yuan6975 opened this issue Oct 5, 2024 · 0 comments

Comments

@yuan6975
Copy link

yuan6975 commented Oct 5, 2024

Device Information | 设备信息

  • SOC: [e.g. S095x3]
  • Model [e.g. M401A]

OpenWrt Version | 系统版本

  • Kernel Version: [e.g. 6.6.43]
  • Release: [e.g. official/lede/immortalwrt]

Describe the bug | 问题描述
All issues will only remain open for one week to prioritize resolving them.
After that period, they will be closed but can still continue to be discussed in the thread.
所有的问题都只保留一周的开启状态供重点关注解决,超时将关闭,但仍然可以长期继续跟帖讨论。

Please provide a detailed description of the issue and, if necessary, attach a screenshot.
详细描述问题,并在必要时附上屏幕截图。

在btrfs文件系统下创建swap之后出现的问题 cpu直接100% 负载直接拉到了90多 我两个s905l3a盒子都是在用新的方式启动swap之后出现的这个问题, 下面是内核日志开始出问题的位置

[208560.275403] INFO: task kthreadd:2 blocked for more than 120 seconds.
[208560.277302] Tainted: G C 6.6.43-flippy-91+ #87
[208560.283751] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[208560.291611] task:kthreadd state:D stack:0 pid:2 ppid:0 flags:0x00000008
[208560.291634] Call trace:
[208560.291638] __switch_to+0x164/0x270
[208560.291654] __schedule+0x5c8/0x850
[208560.291661] schedule+0x60/0x98
[208560.291666] io_schedule+0x38/0xcc
[208560.291672] rq_qos_wait+0xe0/0x138
[208560.291682] wbt_wait+0xc8/0x18c
[208560.291690] __rq_qos_throttle+0x3c/0x50
[208560.291698] blk_mq_submit_bio+0x1b0/0x768
[208560.291707] __submit_bio+0x10c/0x210
[208560.291715] submit_bio_noacct_nocheck+0x1e8/0x2ac
[208560.291723] submit_bio_noacct+0x22c/0x3ec
[208560.291731] submit_bio+0x8c/0xa0
[208560.291738] __swap_writepage+0x484/0x54c
[208560.291748] swap_writepage+0x84/0xe0
[208560.291755] shrink_folio_list+0x968/0xe38
[208560.291765] evict_folios+0x1264/0x1734
[208560.291771] try_to_shrink_lruvec+0x21c/0x31c
[208560.291778] shrink_one+0xc4/0x254
[208560.291784] shrink_node+0xb70/0xe5c
[208560.291790] do_try_to_free_pages+0x1a4/0x574
[208560.291798] try_to_free_pages+0x2bc/0x4c0
[208560.291807] __alloc_pages_slowpath+0x478/0xbc8
[208560.291814] __alloc_pages+0x25c/0x2dc
[208560.291820] __alloc_pages_bulk+0x2c0/0x5d0
[208560.291826] __vmalloc_node_range+0x310/0x764
[208560.291835] dup_task_struct+0x14c/0x2b0
[208560.291845] copy_process+0x150/0xe48
[208560.291852] kernel_clone+0xa8/0x378
[208560.291860] kernel_thread+0x64/0x8c
[208560.291867] kthreadd+0x18c/0x280
[208560.291874] ret_from_fork+0x10/0x20
[208560.291902] INFO: task kworker/R-write:39 blocked for more than 120 seconds.
[208560.298710] Tainted: G C 6.6.43-flippy-91+ #87
[208560.305119] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[208560.313007] task:kworker/R-write state:D stack:0 pid:39 ppid:2 flags:0x00000008
[208560.313018] Workqueue: writeback wb_workfn (flush-btrfs-3)
[208560.313030] Call trace:
[208560.313034] __switch_to+0x164/0x270
[208560.313040] __schedule+0x5c8/0x850
[208560.313046] schedule+0x60/0x98
[208560.313052] schedule_preempt_disabled+0x10/0x1c
[208560.313058] rwsem_down_read_slowpath+0x250/0x580
[208560.313067] down_read+0x44/0xc0
[208560.313073] btrfs_read_lock_root_node+0x48/0x108
[208560.313083] btrfs_search_slot+0x138/0x8d8
[208560.313090] btrfs_lookup_file_extent+0x4c/0x74
[208560.313100] btrfs_drop_extents+0xd0/0xd5c
[208560.313109] cow_file_range_inline+0x160/0x3ec
[208560.313116] cow_file_range+0x34c/0x568
[208560.313123] btrfs_run_delalloc_range+0x9c/0x3dc
[208560.313130] writepage_delalloc+0x9c/0x134
[208560.313139] extent_writepages+0x4a4/0x7cc
[208560.313147] btrfs_writepages+0x10/0x1c
[208560.313154] do_writepages+0xf8/0x224
[208560.313164] __writeback_single_inode+0x40/0x408
[208560.313173] writeback_sb_inodes+0x38c/0x6d8
[208560.313180] __writeback_inodes_wb+0xb8/0x120
[208560.313187] writeback_inodes_wb+0xa8/0x100
[208560.313193] wb_workfn+0x448/0x500
[208560.313200] process_scheduled_works+0x19c/0x410
[208560.313209] rescuer_thread+0x270/0x518
[208560.313216] kthread+0xe4/0xfc
[208560.313223] ret_from_fork+0x10/0x20
[208560.313240] INFO: task kswapd0:58 blocked for more than 120 seconds.
[208560.319421] Tainted: G C 6.6.43-flippy-91+ #87
[208560.325834] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[208560.333724] task:kswapd0 state:D stack:0 pid:58 ppid:2 flags:0x00000008
[208560.333734] Call trace:
[208560.333737] __switch_to+0x164/0x270
[208560.333744] __schedule+0x5c8/0x850
[208560.333750] schedule+0x60/0x98
[208560.333756] io_schedule+0x38/0xcc
[208560.333761] blk_mq_get_tag+0x1d4/0x344
[208560.333771] __blk_mq_alloc_requests+0x234/0x358
[208560.333779] blk_mq_submit_bio+0x1d4/0x768
[208560.333787] __submit_bio+0x10c/0x210
[208560.333795] submit_bio_noacct_nocheck+0x1e8/0x2ac
[208560.333804] submit_bio_noacct+0x22c/0x3ec
[208560.333811] submit_bio+0x8c/0xa0
[208560.333819] __swap_writepage+0x484/0x54c
[208560.333827] swap_writepage+0x84/0xe0
[208560.333834] shrink_folio_list+0x968/0xe38
[208560.333843] evict_folios+0x1264/0x1734
[208560.333849] try_to_shrink_lruvec+0x21c/0x31c
[208560.333856] shrink_one+0xc4/0x254
[208560.333862] shrink_node+0xb70/0xe5c
[208560.333868] kswapd+0x784/0xddc
[208560.333875] kthread+0xe4/0xfc
[208560.333882] ret_from_fork+0x10/0x20
[208560.333947] INFO: task btrfs-transacti:1504 blocked for more than 120 seconds.
[208560.341003] Tainted: G C 6.6.43-flippy-91+ #87
[208560.347415] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[208560.355302] task:btrfs-transacti state:D stack:0 pid:1504 ppid:2 flags:0x00000008
[208560.355311] Call trace:
[208560.355314] __switch_to+0x164/0x270
[208560.355322] __schedule+0x5c8/0x850
[208560.355328] schedule+0x60/0x98
[208560.355333] schedule_preempt_disabled+0x10/0x1c
[208560.355340] rwsem_down_read_slowpath+0x250/0x580
[208560.355348] down_read+0x44/0xc0
[208560.355354] btrfs_read_lock_root_node+0x48/0x108
[208560.355363] btrfs_search_slot+0x138/0x8d8
[208560.355371] btrfs_lookup_inode+0x48/0x104
[208560.355385] __btrfs_update_delayed_inode+0x68/0x418
[208560.355395] __btrfs_commit_inode_delayed_items+0x6e4/0x73c
[208560.355404] __btrfs_run_delayed_items+0x10c/0x244
[208560.355413] btrfs_commit_transaction+0x368/0x10c0
[208560.355420] transaction_kthread+0x144/0x1e8
[208560.355428] kthread+0xe4/0xfc
[208560.355435] ret_from_fork+0x10/0x20
[208560.355450] INFO: task kworker/R-btrfs:1683 blocked for more than 120 seconds.
[208560.362582] Tainted: G C 6.6.43-flippy-91+ #87
[208560.368992] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[208560.376882] task:kworker/R-btrfs state:D stack:0 pid:1683 ppid:2 flags:0x00000008
[208560.376892] Workqueue: btrfs-endio-write btrfs_work_helper
[208560.376901] Call trace:
[208560.376904] __switch_to+0x164/0x270
[208560.376911] __schedule+0x5c8/0x850
[208560.376917] schedule+0x60/0x98
[208560.376922] schedule_preempt_disabled+0x10/0x1c
[208560.376929] rwsem_down_write_slowpath+0x3a8/0x828
[208560.376936] down_write+0x48/0x64
[208560.376943] btrfs_tree_lock+0x24/0x9c
[208560.376950] btrfs_search_slot+0x270/0x8d8
[208560.376956] btrfs_lookup_file_extent+0x4c/0x74
[208560.376964] btrfs_drop_extents+0xd0/0xd5c
[208560.376973] insert_reserved_file_extent+0xcc/0x320
[208560.376981] btrfs_finish_one_ordered+0x2d0/0x83c
[208560.376988] finish_ordered_fn+0x14/0x20
[208560.376997] btrfs_work_helper+0x128/0x4e8
[208560.377003] process_scheduled_works+0x19c/0x410
[208560.377012] rescuer_thread+0x270/0x518
[208560.377019] kthread+0xe4/0xfc
[208560.377026] ret_from_fork+0x10/0x20
[208560.377037] INFO: task btrfs-transacti:1701 blocked for more than 120 seconds.
[208560.384162] Tainted: G C 6.6.43-flippy-91+ #87
[208560.390575] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[208560.398460] task:btrfs-transacti state:D stack:0 pid:1701 ppid:2 flags:0x00000008
[208560.398469] Call trace:
[208560.398472] __switch_to+0x164/0x270
[208560.398480] __schedule+0x5c8/0x850
[208560.398486] schedule+0x60/0x98
[208560.398491] schedule_preempt_disabled+0x10/0x1c
[208560.398498] rwsem_down_read_slowpath+0x250/0x580
[208560.398505] down_read+0x44/0xc0
[208560.398512] btrfs_read_lock_root_node+0x48/0x108
[208560.398520] btrfs_search_slot+0x138/0x8d8
[208560.398526] btrfs_lookup_inode+0x48/0x104
[208560.398534] __btrfs_update_delayed_inode+0x68/0x418
[208560.398542] __btrfs_commit_inode_delayed_items+0x6e4/0x73c
[208560.398551] __btrfs_run_delayed_items+0x10c/0x244
[208560.398560] btrfs_commit_transaction+0x368/0x10c0
[208560.398567] transaction_kthread+0x144/0x1e8
[208560.398575] kthread+0xe4/0xfc
[208560.398582] ret_from_fork+0x10/0x20
[208560.398613] INFO: task dockerd:7226 blocked for more than 120 seconds.
[208560.405046] Tainted: G C 6.6.43-flippy-91+ #87
[208560.411461] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[208560.419348] task:dockerd state:D stack:0 pid:7226 ppid:1 flags:0x00000001
[208560.419357] Call trace:
[208560.419361] __switch_to+0x164/0x270
[208560.419368] __schedule+0x5c8/0x850
[208560.419374] schedule+0x60/0x98
[208560.419385] schedule_preempt_disabled+0x10/0x1c
[208560.419392] rwsem_down_read_slowpath+0x250/0x580
[208560.419399] down_read+0x44/0xc0
[208560.419406] btrfs_read_lock_root_node+0x48/0x108
[208560.419414] btrfs_search_slot+0x138/0x8d8
[208560.419420] btrfs_lookup_csum+0x60/0x158
[208560.419428] btrfs_lookup_bio_sums+0x27c/0x3dc
[208560.419436] btrfs_submit_bio+0x2d4/0x46c
[208560.419446] btrfs_read_folio+0xb0/0xe0
[208560.419454] filemap_fault+0x3e8/0x720
[208560.419462] __do_fault+0xb8/0x1ac
[208560.419471] handle_mm_fault+0x8f4/0xf4c
[208560.419478] do_page_fault+0x214/0x44c
[208560.419486] do_translation_fault+0x38/0x50
[208560.419494] do_mem_abort+0x40/0xec
[208560.419503] el0_da+0x44/0x74
[208560.419511] el0t_64_sync_handler+0x60/0xfc
[208560.419518] el0t_64_sync+0x1a4/0x1a8
[208560.419541] INFO: task snmpd:8984 blocked for more than 120 seconds.
[208560.425762] Tainted: G C 6.6.43-flippy-91+ #87
[208560.432173] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[208560.440062] task:snmpd state:D stack:0 pid:8984 ppid:1 flags:0x00000000
[208560.440071] Call trace:
[208560.440074] __switch_to+0x164/0x270
[208560.440081] __schedule+0x5c8/0x850
[208560.440086] schedule+0x60/0x98
[208560.440092] io_schedule+0x38/0xcc
[208560.440098] rq_qos_wait+0xe0/0x138
[208560.440106] wbt_wait+0xc8/0x18c
[208560.440114] __rq_qos_throttle+0x3c/0x50
[208560.440121] blk_mq_submit_bio+0x1b0/0x768
[208560.440129] __submit_bio+0x10c/0x210
[208560.440137] submit_bio_noacct_nocheck+0x1e8/0x2ac
[208560.440145] submit_bio_noacct+0x22c/0x3ec
[208560.440153] submit_bio+0x8c/0xa0
[208560.440160] __swap_writepage+0x484/0x54c
[208560.440168] swap_writepage+0x84/0xe0
[208560.440175] shrink_folio_list+0x968/0xe38
[208560.440184] evict_folios+0x1264/0x1734
[208560.440190] try_to_shrink_lruvec+0x21c/0x31c
[208560.440197] shrink_one+0xc4/0x254
[208560.440203] shrink_node+0xb70/0xe5c
[208560.440208] do_try_to_free_pages+0x1a4/0x574
[208560.440217] try_to_free_pages+0x2bc/0x4c0
[208560.440225] __alloc_pages_slowpath+0x478/0xbc8
[208560.440233] __alloc_pages+0x25c/0x2dc
[208560.440239] __folio_alloc+0x1c/0x44
[208560.440244] vma_alloc_zeroed_movable_folio+0x38/0x48
[208560.440253] handle_mm_fault+0x70c/0xf4c
[208560.440261] do_page_fault+0x1a8/0x44c
[208560.440269] do_translation_fault+0x38/0x50
[208560.440277] do_mem_abort+0x40/0xec
[208560.440284] el0_da+0x44/0x74
[208560.440291] el0t_64_sync_handler+0x60/0xfc
[208560.440299] el0t_64_sync+0x1a4/0x1a8
[208560.440343] INFO: task PCDNClient:15423 blocked for more than 120 seconds.
[208560.446996] Tainted: G C 6.6.43-flippy-91+ #87
[208560.453406] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[208560.461295] task:PCDNClient state:D stack:0 pid:15423 ppid:14811 flags:0x00000000
[208560.461304] Call trace:
[208560.461307] __switch_to+0x164/0x270
[208560.461315] __schedule+0x5c8/0x850
[208560.461320] schedule+0x60/0x98
[208560.461326] schedule_preempt_disabled+0x10/0x1c
[208560.461333] rwsem_down_read_slowpath+0x250/0x580
[208560.461341] down_read+0x44/0xc0
[208560.461347] btrfs_read_lock_root_node+0x48/0x108
[208560.461356] btrfs_search_slot+0x138/0x8d8
[208560.461362] btrfs_lookup_file_extent+0x4c/0x74
[208560.461370] btrfs_get_extent+0x14c/0x534
[208560.461377] btrfs_do_readpage+0x2b4/0x6f4
[208560.461385] extent_readahead+0x238/0x308
[208560.461392] btrfs_readahead+0x10/0x1c
[208560.461400] read_pages+0x54/0x290
[208560.461407] page_cache_ra_unbounded+0x9c/0x1a4
[208560.461414] page_cache_ra_order+0x29c/0x2c4
[208560.461420] do_sync_mmap_readahead+0x11c/0x22c
[208560.461428] filemap_fault+0x1c0/0x720
[208560.461435] __do_fault+0xb8/0x1ac
[208560.461442] handle_mm_fault+0x8f4/0xf4c
[208560.461450] do_page_fault+0x214/0x44c
[208560.461457] do_translation_fault+0x38/0x50
[208560.461465] do_mem_abort+0x40/0xec
[208560.461473] el0_da+0x44/0x74
[208560.461480] el0t_64_sync_handler+0x60/0xfc
[208560.461488] el0t_64_sync+0x1a4/0x1a8
[208560.461495] INFO: task PCDNClient:15430 blocked for more than 121 seconds.
[208560.468229] Tainted: G C 6.6.43-flippy-91+ #87
[208560.474641] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[208560.482526] task:PCDNClient state:D stack:0 pid:15430 ppid:14811 flags:0x00000000
[208560.482535] Call trace:
[208560.482538] __switch_to+0x164/0x270
[208560.482545] __schedule+0x5c8/0x850
[208560.482551] schedule+0x60/0x98
[208560.482557] wait_current_trans+0x12c/0x198
[208560.482564] start_transaction+0x22c/0x520
[208560.482574] btrfs_setattr+0x1e8/0x608
[208560.482581] notify_change+0x2f4/0x334
[208560.482590] do_truncate+0x8c/0xd8
[208560.482598] path_openat+0xb30/0xc84
[208560.482607] do_filp_open+0x80/0x124
[208560.482615] do_sys_openat2+0x84/0xe8
[208560.482623] __arm64_sys_openat+0x70/0x98
[208560.482631] invoke_syscall+0x40/0x100
[208560.482640] el0_svc_common+0xa8/0xe0
[208560.482649] do_el0_svc+0x1c/0x28
[208560.482657] el0_svc+0x38/0x64
[208560.482664] el0t_64_sync_handler+0x90/0xfc
[208560.482672] el0t_64_sync+0x1a4/0x1a8
[208560.482678] Future hung task reports are suppressed, see sysctl kernel.hung_task_warnings

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant