Description:
In our Arch Linux part of the upstream systemd CI I recently noticed an uptrend in CPU soft lockups when running one of our tests. This test runs several systemd-nspawn containers in succession and sometimes the underlying VM locks up due to a CPU soft lockup (just to clarify, the topology is: CentOS Stream 8 (baremetal) -> Arch Linux (KVM) -> Arch Linux (nested KVM or QEMU TCG, happens with both) -> nspawn containers).

Some examples:

```
[ 88.654931] watchdog: BUG: soft lockup - CPU#0 stuck for 26s! [systemd:1]
[ 88.655506] Modules linked in: ext4 crc32c_generic crc16 mbcache jbd2 sr_mod cdrom serio_raw atkbd libps2 vivaldi_fmap ata_generic pata_acpi virtio_pci virtio_pci_legacy_dev virtio_pci_modern_dev ata_piix floppy i8042 serio virtio_rng e
[ 88.656219] Unloaded tainted modules: crc32c_intel():1
[ 88.657433] CPU: 0 PID: 1 Comm: systemd Not tainted 5.19.5-arch1-1 #1 bcfbb9540ef8053872a922808497e5b96b6f8fc8
[ 88.657999] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS Arch Linux 1.16.0-3-3 04/01/2014
[ 88.658186] RIP: 0010:___slab_alloc+0x555/0x6b0
[ 88.658692] Code: 73 08 e9 d7 00 00 00 48 89 fb 48 8b 03 48 85 c0 0f 84 b7 fe ff ff 48 63 40 58 4d 8b 8c c5 d8 00 00 00 48 89 c7 4d 85 c9 74 0c <eb> 3e 49 8b 41 08 49 39 45 10 72 4b 48 8d 7b 10 44 3b 63 18 73 c7
[ 88.658692] RSP: 0018:ffffbb3fc0013b58 EFLAGS: 00000282
[ 88.658692] RAX: 0000000000000000 RBX: ffff9cab6e9e41c0 RCX: 0000000000000000
[ 88.658692] RDX: 0000000000000000 RSI: ffff9cab41041280 RDI: 0000000000000000
[ 88.658692] RBP: 0000000000000dc0 R08: ffff9cab6e9e41c0 R09: ffff9cab41041280
[ 88.658692] R10: ffff9cab4623adc8 R11: ffffbb3fc0013c20 R12: 0000000000000002
[ 88.658692] R13: ffff9cab41042a00 R14: 00000000ffffffff R15: ffff9cab6da38140
[ 88.662511] FS: 00007f8020ae6a40(0000) GS:ffff9cab6da00000(0000) knlGS:0000000000000000
[ 88.662511] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 88.662511] CR2: 00007f654f796445 CR3: 000000002f494000 CR4: 00000000000006f0
[ 88.662511] Call Trace:
[ 88.662511] <TASK>
[ 88.662511] ? cpuset_css_alloc+0x2f/0x150
[ 88.662511] ? cgroup_addrm_files+0x320/0x370
[ 88.662511] ? cpuset_css_alloc+0x2f/0x150
[ 88.662511] kmem_cache_alloc_trace+0x29e/0x310
[ 88.665067] cpuset_css_alloc+0x2f/0x150
[ 88.665067] cgroup_apply_control_enable+0x120/0x340
[ 88.665067] cgroup_subtree_control_write+0x37b/0x400
[ 88.665067] kernfs_fop_write_iter+0x122/0x200
[ 88.665067] new_sync_write+0x13a/0x1c0
[ 88.665067] vfs_write+0x1ef/0x270
[ 88.665067] ksys_write+0x6f/0xf0
[ 88.665067] do_syscall_64+0x5f/0x90
[ 88.665067] ? vfs_write+0x178/0x270
[ 88.665067] ? vfs_write+0x178/0x270
[ 88.665067] ? ksys_write+0x6f/0xf0
[ 88.665067] ? syscall_exit_to_user_mode+0x1b/0x40
[ 88.665067] ? do_syscall_64+0x6b/0x90
[ 88.665067] ? do_syscall_64+0x6b/0x90
[ 88.665067] ? do_syscall_64+0x6b/0x90
[ 88.665067] entry_SYSCALL_64_after_hwframe+0x63/0xcd
[ 88.665067] RIP: 0033:0x7f8020f0fe94
[ 88.668682] Code: 15 f9 0e 0e 00 f7 d8 64 89 02 48 c7 c0 ff ff ff ff eb b7 0f 1f 00 f3 0f 1e fa 80 3d 8d 96 0e 00 00 74 13 b8 01 00 00 00 0f 05 <48> 3d 00 f0 ff ff 77 54 c3 0f 1f 00 48 83 ec 28 48 89 54 24 18 48
[ 88.668682] RSP: 002b:00007fff61974728 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 88.668682] RAX: ffffffffffffffda RBX: 0000000000000008 RCX: 00007f8020f0fe94
[ 88.668682] RDX: 0000000000000008 RSI: 000055ef79cdf1c0 RDI: 000000000000003f
[ 88.668682] RBP: 000055ef79cdf1c0 R08: fefefefefefefeff R09: ff736472746f622a
[ 88.668682] R10: 00007f8020ebfe00 R11: 0000000000000202 R12: 0000000000000008
[ 88.668682] R13: 000055ef79cd2130 R14: 0000000000000008 R15: 00007f8020fed940
[ 88.668682] </TASK>
[ 116.654866] watchdog: BUG: soft lockup - CPU#0 stuck for 52s! [systemd:1]
[ 116.655272] Modules linked in: ext4 crc32c_generic crc16 mbcache jbd2 sr_mod cdrom serio_raw atkbd libps2 vivaldi_fmap ata_generic pata_acpi virtio_pci virtio_pci_legacy_dev virtio_pci_modern_dev ata_piix floppy i8042 serio virtio_rng e
[ 116.655401] Unloaded tainted modules: crc32c_intel():1
[ 116.656629] CPU: 0 PID: 1 Comm: systemd Tainted: G L 5.19.5-arch1-1 #1 bcfbb9540ef8053872a922808497e5b96b6f8fc8
[ 116.657113] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS Arch Linux 1.16.0-3-3 04/01/2014
[ 116.657487] RIP: 0010:___slab_alloc+0x555/0x6b0
[ 116.657733] Code: 73 08 e9 d7 00 00 00 48 89 fb 48 8b 03 48 85 c0 0f 84 b7 fe ff ff 48 63 40 58 4d 8b 8c c5 d8 00 00 00 48 89 c7 4d 85 c9 74 0c <eb> 3e 49 8b 41 08 49 39 45 10 72 4b 48 8d 7b 10 44 3b 63 18 73 c7
[ 116.658185] RSP: 0018:ffffbb3fc0013b58 EFLAGS: 00000282
[ 116.658185] RAX: 0000000000000000 RBX: ffff9cab6e9e41c0 RCX: 0000000000000000
[ 116.659383] RDX: 0000000000000000 RSI: ffff9cab41041280 RDI: 0000000000000000
[ 116.659383] RBP: 0000000000000dc0 R08: ffff9cab6e9e41c0 R09: ffff9cab41041280
[ 116.659383] R10: ffff9cab4623adc8 R11: ffffbb3fc0013c20 R12: 0000000000000002
[ 116.659383] R13: ffff9cab41042a00 R14: 00000000ffffffff R15: ffff9cab6da38140
[ 116.659383] FS: 00007f8020ae6a40(0000) GS:ffff9cab6da00000(0000) knlGS:0000000000000000
[ 116.659383] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 116.659383] CR2: 00007f654f796445 CR3: 000000002f494000 CR4: 00000000000006f0
[ 116.661746] Call Trace:
[ 116.661746] <TASK>
[ 116.661746] ? cpuset_css_alloc+0x2f/0x150
[ 116.661746] ? cgroup_addrm_files+0x320/0x370
[ 116.661746] ? cpuset_css_alloc+0x2f/0x150
[ 116.661746] kmem_cache_alloc_trace+0x29e/0x310
[ 116.661746] cpuset_css_alloc+0x2f/0x150
[ 116.661746] cgroup_apply_control_enable+0x120/0x340
[ 116.661746] cgroup_subtree_control_write+0x37b/0x400
[ 116.661746] kernfs_fop_write_iter+0x122/0x200
[ 116.661746] new_sync_write+0x13a/0x1c0
[ 116.661746] vfs_write+0x1ef/0x270
[ 116.661746] ksys_write+0x6f/0xf0
[ 116.661746] do_syscall_64+0x5f/0x90
[ 116.661746] ? vfs_write+0x178/0x270
[ 116.661746] ? vfs_write+0x178/0x270
[ 116.661746] ? ksys_write+0x6f/0xf0
[ 116.661746] ? syscall_exit_to_user_mode+0x1b/0x40
[ 116.665695] ? do_syscall_64+0x6b/0x90
[ 116.665695] ? do_syscall_64+0x6b/0x90
[ 116.665695] ? do_syscall_64+0x6b/0x90
[ 116.665695] entry_SYSCALL_64_after_hwframe+0x63/0xcd
[ 116.665695] RIP: 0033:0x7f8020f0fe94
[ 116.665695] Code: 15 f9 0e 0e 00 f7 d8 64 89 02 48 c7 c0 ff ff ff ff eb b7 0f 1f 00 f3 0f 1e fa 80 3d 8d 96 0e 00 00 74 13 b8 01 00 00 00 0f 05 <48> 3d 00 f0 ff ff 77 54 c3 0f 1f 00 48 83 ec 28 48 89 54 24 18 48
[ 116.665695] RSP: 002b:00007fff61974728 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 116.665695] RAX: ffffffffffffffda RBX: 0000000000000008 RCX: 00007f8020f0fe94
[ 116.665695] RDX: 0000000000000008 RSI: 000055ef79cdf1c0 RDI: 000000000000003f
[ 116.668853] RBP: 000055ef79cdf1c0 R08: fefefefefefefeff R09: ff736472746f622a
[ 116.668853] R10: 00007f8020ebfe00 R11: 0000000000000202 R12: 0000000000000008
[ 116.668853] R13: 000055ef79cd2130 R14: 0000000000000008 R15: 00007f8020fed940
[ 116.668853] </TASK>
[ 122.198410] rcu: INFO: rcu_preempt detected stalls on CPUs/tasks:
[ 122.198943] (detected by 3, t=18002 jiffies, g=18269, q=147 ncpus=4)
[ 122.199383] rcu: All QSes seen, last rcu_preempt kthread activity 18002 (4294913871-4294895869), jiffies_till_next_fqs=2, root ->qsmask 0x0
[ 122.200142] rcu: rcu_preempt kthread starved for 18002 jiffies! g18269 f0x2 RCU_GP_WAIT_FQS(5) ->state=0x0 ->cpu=0
[ 122.200601] rcu: Unless rcu_preempt kthread gets sufficient CPU time, OOM is now expected behavior.
[ 122.200969] rcu: RCU grace-period kthread stack dump:
[ 122.201280] task:rcu_preempt state:R running task stack: 0 pid: 16 ppid: 2 flags:0x00004000
[ 122.201590] Call Trace:
[ 122.201590] <TASK>
[ 122.201590] ? rcu_gp_cleanup+0x3c0/0x3c0
[ 122.201590] __schedule+0x356/0x11a0
[ 122.201590] ? __mod_timer+0x289/0x3b0
[ 122.201590] ? rcu_gp_cleanup+0x3c0/0x3c0
[ 122.201590] schedule+0x5e/0xd0
[ 122.201590] schedule_timeout+0x98/0x150
[ 122.201590] ? __bpf_trace_tick_stop+0x10/0x10
[ 122.201590] rcu_gp_fqs_loop+0x113/0x3c0
[ 122.201590] rcu_gp_kthread+0xb0/0x170
[ 122.201590] kthread+0xde/0x110
[ 122.201590] ? kthread_complete_and_exit+0x20/0x20
[ 122.201590] ret_from_fork+0x22/0x30
[ 122.201590] </TASK>
[ 122.201590] rcu: Stack dump where RCU GP kthread last ran:
[ 122.201590] Sending NMI from CPU 3 to CPUs 0:
[ 122.206283] NMI backtrace for cpu 0
[ 122.207931] CPU: 0 PID: 1 Comm: systemd Tainted: G L 5.19.5-arch1-1 #1 bcfbb9540ef8053872a922808497e5b96b6f8fc8
[ 122.208217] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS Arch Linux 1.16.0-3-3 04/01/2014
[ 122.208217] RIP: 0010:asm_exc_int3+0x0/0x40
[ 122.208217] Code: e9 d5 0a 00 00 0f 1f 44 00 00 f3 0f 1e fa 0f 1f 00 fc 6a ff e8 a1 09 00 00 48 89 c4 48 89 e7 e8 c6 77 fd ff e9 b1 0a 00 00 90 <f3> 0f 1e fa 0f 1f 00 fc 6a ff f6 44 24 10 03 75 18 ff 74 24 28 ff
[ 122.208217] RSP: 0018:ffffbb3fc0013b28 EFLAGS: 00000082
[ 122.208217] RAX: 0000000000000000 RBX: ffff9cab6e9e41c0 RCX: 0000000000000000
[ 122.208217] RDX: 0000000000000000 RSI: ffff9cab41041280 RDI: 0000000000000000
[ 122.208217] RBP: 0000000000000dc0 R08: ffff9cab6e9e41c0 R09: ffff9cab41041280
[ 122.208217] R10: ffff9cab4623adc8 R11: ffffbb3fc0013c20 R12: 0000000000000002
[ 122.208217] R13: ffff9cab41042a00 R14: 00000000ffffffff R15: ffff9cab6da38140
[ 122.208217] FS: 00007f8020ae6a40(0000) GS:ffff9cab6da00000(0000) knlGS:0000000000000000
[ 122.208217] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 122.208217] CR2: 00007f654f796445 CR3: 000000002f494000 CR4: 00000000000006f0
[ 122.208217] Call Trace:
[ 122.208217] <TASK>
[ 122.208217] RIP: 0010:___slab_alloc+0x556/0x6b0
[ 122.208217] Code: 08 e9 d7 00 00 00 48 89 fb 48 8b 03 48 85 c0 0f 84 b7 fe ff ff 48 63 40 58 4d 8b 8c c5 d8 00 00 00 48 89 c7 4d 85 c9 74 0c eb <3e> 49 8b 41 08 49 39 45 10 72 4b 48 8d 7b 10 44 3b 63 18 73 c7 31
[ 122.208217] RSP: 0018:ffffbb3fc0013b58 EFLAGS: 00000282
[ 122.208217] ? ___slab_alloc+0x50d/0x6b0
[ 122.208217] ? cpuset_css_alloc+0x2f/0x150
[ 122.208217] ? cgroup_addrm_files+0x320/0x370
[ 122.208217] ? cpuset_css_alloc+0x2f/0x150
[ 122.208217] kmem_cache_alloc_trace+0x29e/0x310
[ 122.208217] cpuset_css_alloc+0x2f/0x150
[ 122.208217] cgroup_apply_control_enable+0x120/0x340
[ 122.208217] cgroup_subtree_control_write+0x37b/0x400
[ 122.208217] kernfs_fop_write_iter+0x122/0x200
[ 122.208217] new_sync_write+0x13a/0x1c0
[ 122.208217] vfs_write+0x1ef/0x270
[ 122.208217] ksys_write+0x6f/0xf0
[ 122.208217] do_syscall_64+0x5f/0x90
[ 122.208217] ? vfs_write+0x178/0x270
[ 122.208217] ? vfs_write+0x178/0x270
[ 122.208217] ? ksys_write+0x6f/0xf0
[ 122.208217] ? syscall_exit_to_user_mode+0x1b/0x40
[ 122.208217] ? do_syscall_64+0x6b/0x90
[ 122.208217] ? do_syscall_64+0x6b/0x90
[ 122.208217] ? do_syscall_64+0x6b/0x90
[ 122.208217] entry_SYSCALL_64_after_hwframe+0x63/0xcd
[ 122.208217] RIP: 0033:0x7f8020f0fe94
[ 122.208217] Code: 15 f9 0e 0e 00 f7 d8 64 89 02 48 c7 c0 ff ff ff ff eb b7 0f 1f 00 f3 0f 1e fa 80 3d 8d 96 0e 00 00 74 13 b8 01 00 00 00 0f 05 <48> 3d 00 f0 ff ff 77 54 c3 0f 1f 00 48 83 ec 28 48 89 54 24 18 48
[ 122.208217] RSP: 002b:00007fff61974728 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 122.208217] RAX: ffffffffffffffda RBX: 0000000000000008 RCX: 00007f8020f0fe94
[ 122.208217] RDX: 0000000000000008 RSI: 000055ef79cdf1c0 RDI: 000000000000003f
[ 122.208217] RBP: 000055ef79cdf1c0 R08: fefefefefefefeff R09: ff736472746f622a
[ 122.208217] R10: 00007f8020ebfe00 R11: 0000000000000202 R12: 0000000000000008
[ 122.208217] R13: 000055ef79cd2130 R14: 0000000000000008 R15: 00007f8020fed940
[ 122.208217] </TASK>
[ 122.208217] INFO: NMI handler (nmi_cpu_backtrace_handler) took too long to run: 6.624 msecs
[ 148.654860] watchdog: BUG: soft lockup - CPU#0 stuck for 82s! [systemd:1]
[ 148.655260] Modules linked in: ext4 crc32c_generic crc16 mbcache jbd2 sr_mod cdrom serio_raw atkbd libps2 vivaldi_fmap ata_generic pata_acpi virtio_pci virtio_pci_legacy_dev virtio_pci_modern_dev ata_piix floppy i8042 serio virtio_rng e
[ 148.655390] Unloaded tainted modules: crc32c_intel():1
[ 148.656417] CPU: 0 PID: 1 Comm: systemd Tainted: G L 5.19.5-arch1-1 #1 bcfbb9540ef8053872a922808497e5b96b6f8fc8
[ 148.656875] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS Arch Linux 1.16.0-3-3 04/01/2014
[ 148.657223] RIP: 0010:___slab_alloc+0x555/0x6b0
[ 148.657427] Code: 73 08 e9 d7 00 00 00 48 89 fb 48 8b 03 48 85 c0 0f 84 b7 fe ff ff 48 63 40 58 4d 8b 8c c5 d8 00 00 00 48 89 c7 4d 85 c9 74 0c <eb> 3e 49 8b 41 08 49 39 45 10 72 4b 48 8d 7b 10 44 3b 63 18 73 c7
[ 148.658054] RSP: 0018:ffffbb3fc0013b58 EFLAGS: 00000282
[ 148.658177] RAX: 0000000000000000 RBX: ffff9cab6e9e41c0 RCX: 0000000000000000
[ 148.659005] RDX: 0000000000000000 RSI: ffff9cab41041280 RDI: 0000000000000000
[ 148.659129] RBP: 0000000000000dc0 R08: ffff9cab6e9e41c0 R09: ffff9cab41041280
[ 148.659129] R10: ffff9cab4623adc8 R11: ffffbb3fc0013c20 R12: 0000000000000002
[ 148.659129] R13: ffff9cab41042a00 R14: 00000000ffffffff R15: ffff9cab6da38140
[ 148.659129] FS: 00007f8020ae6a40(0000) GS:ffff9cab6da00000(0000) knlGS:0000000000000000
[ 148.659129] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 148.659129] CR2: 00007f654f796445 CR3: 000000002f494000 CR4: 00000000000006f0
[ 148.659129] Call Trace:
[ 148.659129] <TASK>
[ 148.659129] ? cpuset_css_alloc+0x2f/0x150
[ 148.659129] ? cgroup_addrm_files+0x320/0x370
[ 148.662160] ? cpuset_css_alloc+0x2f/0x150
[ 148.662160] kmem_cache_alloc_trace+0x29e/0x310
[ 148.662160] cpuset_css_alloc+0x2f/0x150
[ 148.662160] cgroup_apply_control_enable+0x120/0x340
[ 148.662160] cgroup_subtree_control_write+0x37b/0x400
[ 148.662160] kernfs_fop_write_iter+0x122/0x200
[ 148.662160] new_sync_write+0x13a/0x1c0
[ 148.662160] vfs_write+0x1ef/0x270
[ 148.662160] ksys_write+0x6f/0xf0
[ 148.662160] do_syscall_64+0x5f/0x90
[ 148.662160] ? vfs_write+0x178/0x270
[ 148.662160] ? vfs_write+0x178/0x270
[ 148.662160] ? ksys_write+0x6f/0xf0
[ 148.662160] ? syscall_exit_to_user_mode+0x1b/0x40
[ 148.662160] ? do_syscall_64+0x6b/0x90
[ 148.662160] ? do_syscall_64+0x6b/0x90
[ 148.665743] ? do_syscall_64+0x6b/0x90
[ 148.665743] entry_SYSCALL_64_after_hwframe+0x63/0xcd
[ 148.665743] RIP: 0033:0x7f8020f0fe94
[ 148.665743] Code: 15 f9 0e 0e 00 f7 d8 64 89 02 48 c7 c0 ff ff ff ff eb b7 0f 1f 00 f3 0f 1e fa 80 3d 8d 96 0e 00 00 74 13 b8 01 00 00 00 0f 05 <48> 3d 00 f0 ff ff 77 54 c3 0f 1f 00 48 83 ec 28 48 89 54 24 18 48
[ 148.665743] RSP: 002b:00007fff61974728 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 148.665743] RAX: ffffffffffffffda RBX: 0000000000000008 RCX: 00007f8020f0fe94
[ 148.665743] RDX: 0000000000000008 RSI: 000055ef79cdf1c0 RDI: 000000000000003f
[ 148.668299] RBP: 000055ef79cdf1c0 R08: fefefefefefefeff R09: ff736472746f622a
[ 148.668846] R10: 00007f8020ebfe00 R11: 0000000000000202 R12: 0000000000000008
[ 148.668846] R13: 000055ef79cd2130 R14: 0000000000000008 R15: 00007f8020fed940
[ 148.668846] </TASK>
[ 176.654938] watchdog: BUG: soft lockup - CPU#0 stuck for 108s! [systemd:1]
[ 176.655507] Modules linked in: ext4 crc32c_generic crc16 mbcache jbd2 sr_mod cdrom serio_raw atkbd libps2 vivaldi_fmap ata_generic pata_acpi virtio_pci virtio_pci_legacy_dev virtio_pci_modern_dev ata_piix floppy i8042 serio virtio_rng e
[ 176.655633] Unloaded tainted modules: crc32c_intel():1
[ 176.656523] CPU: 0 PID: 1 Comm: systemd Tainted: G L 5.19.5-arch1-1 #1 bcfbb9540ef8053872a922808497e5b96b6f8fc8
[ 176.656865] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS Arch Linux 1.16.0-3-3 04/01/2014
[ 176.657188] RIP: 0010:___slab_alloc+0x555/0x6b0
[ 176.657387] Code: 73 08 e9 d7 00 00 00 48 89 fb 48 8b 03 48 85 c0 0f 84 b7 fe ff ff 48 63 40 58 4d 8b 8c c5 d8 00 00 00 48 89 c7 4d 85 c9 74 0c <eb> 3e 49 8b 41 08 49 39 45 10 72 4b 48 8d 7b 10 44 3b 63 18 73 c7
[ 176.657988] RSP: 0018:ffffbb3fc0013b58 EFLAGS: 00000282
[ 176.658193] RAX: 0000000000000000 RBX: ffff9cab6e9e41c0 RCX: 0000000000000000
[ 176.658627] RDX: 0000000000000000 RSI: ffff9cab41041280 RDI: 0000000000000000
[ 176.658627] RBP: 0000000000000dc0 R08: ffff9cab6e9e41c0 R09: ffff9cab41041280
[ 176.658627] R10: ffff9cab4623adc8 R11: ffffbb3fc0013c20 R12: 0000000000000002
[ 176.658627] R13: ffff9cab41042a00 R14: 00000000ffffffff R15: ffff9cab6da38140
[ 176.658627] FS: 00007f8020ae6a40(0000) GS:ffff9cab6da00000(0000) knlGS:0000000000000000
[ 176.658627] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 176.658627] CR2: 00007f654f796445 CR3: 000000002f494000 CR4: 00000000000006f0
[ 176.658627] Call Trace:
[ 176.658627] <TASK>
[ 176.658627] ? cpuset_css_alloc+0x2f/0x150
[ 176.658627] ? cgroup_addrm_files+0x320/0x370
[ 176.658627] ? cpuset_css_alloc+0x2f/0x150
[ 176.658627] kmem_cache_alloc_trace+0x29e/0x310
[ 176.658627] cpuset_css_alloc+0x2f/0x150
[ 176.658627] cgroup_apply_control_enable+0x120/0x340
[ 176.658627] cgroup_subtree_control_write+0x37b/0x400
[ 176.662368] kernfs_fop_write_iter+0x122/0x200
[ 176.662368] new_sync_write+0x13a/0x1c0
[ 176.662368] vfs_write+0x1ef/0x270
[ 176.662368] ksys_write+0x6f/0xf0
[ 176.662368] do_syscall_64+0x5f/0x90
[ 176.662368] ? vfs_write+0x178/0x270
[ 176.662368] ? vfs_write+0x178/0x270
[ 176.662368] ? ksys_write+0x6f/0xf0
[ 176.662368] ? syscall_exit_to_user_mode+0x1b/0x40
[ 176.662368] ? do_syscall_64+0x6b/0x90
[ 176.662368] ? do_syscall_64+0x6b/0x90
[ 176.662368] ? do_syscall_64+0x6b/0x90
[ 176.662368] entry_SYSCALL_64_after_hwframe+0x63/0xcd
[ 176.665151] RIP: 0033:0x7f8020f0fe94
[ 176.665151] Code: 15 f9 0e 0e 00 f7 d8 64 89 02 48 c7 c0 ff ff ff ff eb b7 0f 1f 00 f3 0f 1e fa 80 3d 8d 96 0e 00 00 74 13 b8 01 00 00 00 0f 05 <48> 3d 00 f0 ff ff 77 54 c3 0f 1f 00 48 83 ec 28 48 89 54 24 18 48
[ 176.665151] RSP: 002b:00007fff61974728 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 176.665151] RAX: ffffffffffffffda RBX: 0000000000000008 RCX: 00007f8020f0fe94
[ 176.665151] RDX: 0000000000000008 RSI: 000055ef79cdf1c0 RDI: 000000000000003f
[ 176.665151] RBP: 000055ef79cdf1c0 R08: fefefefefefefeff R09: ff736472746f622a
[ 176.665151] R10: 00007f8020ebfe00 R11: 0000000000000202 R12: 0000000000000008
[ 176.665151] R13: 000055ef79cd2130 R14: 0000000000000008 R15: 00007f8020fed940
[ 176.665151] </TASK>
[ 204.654852] watchdog: BUG: soft lockup - CPU#0 stuck for 134s! [systemd:1]
[ 204.655338] Modules linked in: ext4 crc32c_generic crc16 mbcache jbd2 sr_mod cdrom serio_raw atkbd libps2 vivaldi_fmap ata_generic pata_acpi virtio_pci virtio_pci_legacy_dev virtio_pci_modern_dev ata_piix floppy i8042 serio virtio_rng e
[ 204.655461] Unloaded tainted modules: crc32c_intel():1
[ 204.656203] CPU: 0 PID: 1 Comm: systemd Tainted: G L 5.19.5-arch1-1 #1 bcfbb9540ef8053872a922808497e5b96b6f8fc8
[ 204.656517] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS Arch Linux 1.16.0-3-3 04/01/2014
[ 204.656761] RIP: 0010:___slab_alloc+0x555/0x6b0
[ 204.656911] Code: 73 08 e9 d7 00 00 00 48 89 fb 48 8b 03 48 85 c0 0f 84 b7 fe ff ff 48 63 40 58 4d 8b 8c c5 d8 00 00 00 48 89 c7 4d 85 c9 74 0c <eb> 3e 49 8b 41 08 49 39 45 10 72 4b 48 8d 7b 10 44 3b 63 18 73 c7
[ 204.657380] RSP: 0018:ffffbb3fc0013b58 EFLAGS: 00000282
[ 204.657526] RAX: 0000000000000000 RBX: ffff9cab6e9e41c0 RCX: 0000000000000000
[ 204.657710] RDX: 0000000000000000 RSI: ffff9cab41041280 RDI: 0000000000000000
[ 204.657887] RBP: 0000000000000dc0 R08: ffff9cab6e9e41c0 R09: ffff9cab41041280
[ 204.658072] R10: ffff9cab4623adc8 R11: ffffbb3fc0013c20 R12: 0000000000000002
[ 204.658469] R13: ffff9cab41042a00 R14: 00000000ffffffff R15: ffff9cab6da38140
[ 204.658719] FS: 00007f8020ae6a40(0000) GS:ffff9cab6da00000(0000) knlGS:0000000000000000
[ 204.658719] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 204.658719] CR2: 00007f654f796445 CR3: 000000002f494000 CR4: 00000000000006f0
[ 204.658719] Call Trace:
[ 204.658719] <TASK>
[ 204.658719] ? cpuset_css_alloc+0x2f/0x150
[ 204.658719] ? cgroup_addrm_files+0x320/0x370
[ 204.658719] ? cpuset_css_alloc+0x2f/0x150
[ 204.658719] kmem_cache_alloc_trace+0x29e/0x310
[ 204.658719] cpuset_css_alloc+0x2f/0x150
[ 204.658719] cgroup_apply_control_enable+0x120/0x340
[ 204.658719] cgroup_subtree_control_write+0x37b/0x400
[ 204.658719] kernfs_fop_write_iter+0x122/0x200
[ 204.658719] new_sync_write+0x13a/0x1c0
[ 204.658719] vfs_write+0x1ef/0x270
[ 204.658719] ksys_write+0x6f/0xf0
[ 204.658719] do_syscall_64+0x5f/0x90
[ 204.658719] ? vfs_write+0x178/0x270
[ 204.658719] ? vfs_write+0x178/0x270
[ 204.658719] ? ksys_write+0x6f/0xf0
[ 204.658719] ? syscall_exit_to_user_mode+0x1b/0x40
[ 204.658719] ? do_syscall_64+0x6b/0x90
[ 204.662054] ? do_syscall_64+0x6b/0x90
[ 204.662054] ? do_syscall_64+0x6b/0x90
[ 204.662054] entry_SYSCALL_64_after_hwframe+0x63/0xcd
[ 204.662054] RIP: 0033:0x7f8020f0fe94
[ 204.662054] Code: 15 f9 0e 0e 00 f7 d8 64 89 02 48 c7 c0 ff ff ff ff eb b7 0f 1f 00 f3 0f 1e fa 80 3d 8d 96 0e 00 00 74 13 b8 01 00 00 00 0f 05 <48> 3d 00 f0 ff ff 77 54 c3 0f 1f 00 48 83 ec 28 48 89 54 24 18 48
[ 204.662054] RSP: 002b:00007fff61974728 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 204.662054] RAX: ffffffffffffffda RBX: 0000000000000008 RCX: 00007f8020f0fe94
[ 204.662054] RDX: 0000000000000008 RSI: 000055ef79cdf1c0 RDI: 000000000000003f
[ 204.662054] RBP: 000055ef79cdf1c0 R08: fefefefefefefeff R09: ff736472746f622a
[ 204.662054] R10: 00007f8020ebfe00 R11: 0000000000000202 R12: 0000000000000008
[ 204.662054] R13: 000055ef79cd2130 R14: 0000000000000008 R15: 00007f8020fed940
[ 204.662054] </TASK>
[ 232.654936] watchdog: BUG: soft lockup - CPU#0 stuck for 160s! [systemd:1]
[ 232.655529] Modules linked in: ext4 crc32c_generic crc16 mbcache jbd2 sr_mod cdrom serio_raw atkbd libps2 vivaldi_fmap ata_generic pata_acpi virtio_pci virtio_pci_legacy_dev virtio_pci_modern_dev ata_piix floppy i8042 serio virtio_rng e
[ 232.655661] Unloaded tainted modules: crc32c_intel():1
[ 232.656380] CPU: 0 PID: 1 Comm: systemd Tainted: G L 5.19.5-arch1-1 #1 bcfbb9540ef8053872a922808497e5b96b6f8fc8
[ 232.656682] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS Arch Linux 1.16.0-3-3 04/01/2014
[ 232.656921] RIP: 0010:___slab_alloc+0x555/0x6b0
[ 232.657072] Code: 73 08 e9 d7 00 00 00 48 89 fb 48 8b 03 48 85 c0 0f 84 b7 fe ff ff 48 63 40 58 4d 8b 8c c5 d8 00 00 00 48 89 c7 4d 85 c9 74 0c <eb> 3e 49 8b 41 08 49 39 45 10 72 4b 48 8d 7b 10 44 3b 63 18 73 c7
[ 232.657541] RSP: 0018:ffffbb3fc0013b58 EFLAGS: 00000282
[ 232.657703] RAX: 0000000000000000 RBX: ffff9cab6e9e41c0 RCX: 0000000000000000
[ 232.657885] RDX: 0000000000000000 RSI: ffff9cab41041280 RDI: 0000000000000000
[ 232.658065] RBP: 0000000000000dc0 R08: ffff9cab6e9e41c0 R09: ffff9cab41041280
[ 232.658255] R10: ffff9cab4623adc8 R11: ffffbb3fc0013c20 R12: 0000000000000002
[ 232.658541] R13: ffff9cab41042a00 R14: 00000000ffffffff R15: ffff9cab6da38140
[ 232.658541] FS: 00007f8020ae6a40(0000) GS:ffff9cab6da00000(0000) knlGS:0000000000000000
[ 232.658541] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 232.658541] CR2: 00007f654f796445 CR3: 000000002f494000 CR4: 00000000000006f0
[ 232.658541] Call Trace:
[ 232.658541] <TASK>
[ 232.658541] ? cpuset_css_alloc+0x2f/0x150
[ 232.658541] ? cgroup_addrm_files+0x320/0x370
[ 232.658541] ? cpuset_css_alloc+0x2f/0x150
[ 232.658541] kmem_cache_alloc_trace+0x29e/0x310
[ 232.658541] cpuset_css_alloc+0x2f/0x150
[ 232.658541] cgroup_apply_control_enable+0x120/0x340
[ 232.658541] cgroup_subtree_control_write+0x37b/0x400
[ 232.658541] kernfs_fop_write_iter+0x122/0x200
[ 232.658541] new_sync_write+0x13a/0x1c0
[ 232.658541] vfs_write+0x1ef/0x270
[ 232.658541] ksys_write+0x6f/0xf0
[ 232.658541] do_syscall_64+0x5f/0x90
[ 232.658541] ? vfs_write+0x178/0x270
[ 232.658541] ? vfs_write+0x178/0x270
[ 232.658541] ? ksys_write+0x6f/0xf0
[ 232.658541] ? syscall_exit_to_user_mode+0x1b/0x40
[ 232.661913] ? do_syscall_64+0x6b/0x90
[ 232.661913] ? do_syscall_64+0x6b/0x90
[ 232.661913] ? do_syscall_64+0x6b/0x90
[ 232.661913] entry_SYSCALL_64_after_hwframe+0x63/0xcd
[ 232.661913] RIP: 0033:0x7f8020f0fe94
[ 232.661913] Code: 15 f9 0e 0e 00 f7 d8 64 89 02 48 c7 c0 ff ff ff ff eb b7 0f 1f 00 f3 0f 1e fa 80 3d 8d 96 0e 00 00 74 13 b8 01 00 00 00 0f 05 <48> 3d 00 f0 ff ff 77 54 c3 0f 1f 00 48 83 ec 28 48 89 54 24 18 48
[ 232.661913] RSP: 002b:00007fff61974728 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 232.661913] RAX: ffffffffffffffda RBX: 0000000000000008 RCX: 00007f8020f0fe94
[ 232.661913] RDX: 0000000000000008 RSI: 000055ef79cdf1c0 RDI: 000000000000003f
[ 232.661913] RBP: 000055ef79cdf1c0 R08: fefefefefefefeff R09: ff736472746f622a
[ 232.661913] R10: 00007f8020ebfe00 R11: 0000000000000202 R12: 0000000000000008
[ 232.661913] R13: 000055ef79cd2130 R14: 0000000000000008 R15: 00007f8020fed940
[ 232.661913] </TASK>
[ 245.669022] INFO: task khugepaged:49 blocked for more than 122 seconds.
[ 245.669579] Tainted: G L 5.19.5-arch1-1 #1
[ 245.669901] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 245.670296] task:khugepaged state:D stack: 0 pid: 49 ppid: 2 flags:0x00004000
[ 245.670765] Call Trace:
[ 245.670913] <TASK>
[ 245.671055] __schedule+0x356/0x11a0
[ 245.671305] ? psi_task_switch+0xc3/0x1f0
[ 245.671787] ? native_load_gs_index+0x19/0x30
[ 245.672039] ? finish_task_switch.isra.0+0x90/0x2d0
[ 245.672307] schedule+0x5e/0xd0
[ 245.672484] schedule_timeout+0x11c/0x150
[ 245.672689] __wait_for_common+0x94/0x1d0
[ 245.672909] ? usleep_range_state+0x90/0x90
[ 245.673187] __flush_work.isra.0+0x171/0x230
[ 245.673408] ? flush_workqueue_prep_pwqs+0x110/0x110
[ 245.673671] __lru_add_drain_all+0x160/0x1f0
[ 245.673910] khugepaged+0x75/0x2200
[ 245.674097] ? __schedule+0x35e/0x11a0
[ 245.674284] ? check_preempt_curr+0x61/0x70
[ 245.674486] ? try_to_wake_up+0x23e/0x550
[ 245.674668] ? cpuacct_percpu_seq_show+0x20/0x20
[ 245.675159] ? collapse_pte_mapped_thp+0x3c0/0x3c0
[ 245.675419] kthread+0xde/0x110
[ 245.675612] ? kthread_complete_and_exit+0x20/0x20
[ 245.675868] ret_from_fork+0x22/0x30
[ 245.676106] </TASK>
[ 245.676329] INFO: task 8:208 blocked for more than 122 seconds.
[ 245.676648] Tainted: G L 5.19.5-arch1-1 #1
[ 245.676929] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 245.677283] task:8 state:D stack: 0 pid: 208 ppid: 1 flags:0x00004002
[ 245.677660] Call Trace:
[ 245.677779] <TASK>
[ 245.677905] __schedule+0x356/0x11a0
[ 245.678126] ? asm_sysvec_apic_timer_interrupt+0x1a/0x20
[ 245.678772] schedule+0x5e/0xd0
[ 245.678960] schedule_preempt_disabled+0x15/0x30
[ 245.679192] __mutex_lock.constprop.0+0x461/0x6e0
[ 245.679438] ? kmem_cache_alloc_trace+0x16b/0x310
[ 245.679731] proc_cgroup_show+0x4c/0x3f0
[ 245.679990] proc_single_show+0x54/0xc0
[ 245.680201] seq_read_iter+0x126/0x450
[ 245.680423] seq_read+0x119/0x150
[ 245.680595] vfs_read+0x9f/0x190
[ 245.680794] ksys_read+0x6f/0xf0
[ 245.680950] do_syscall_64+0x5f/0x90
[ 245.681137] ? syscall_exit_to_user_mode+0x1b/0x40
[ 245.681385] ? do_syscall_64+0x6b/0x90
[ 245.681803] ? syscall_exit_to_user_mode+0x1b/0x40
[ 245.682073] ? do_syscall_64+0x6b/0x90
[ 245.682278] ? do_syscall_64+0x6b/0x90
[ 245.682463] entry_SYSCALL_64_after_hwframe+0x63/0xcd
[ 245.682752] RIP: 0033:0x7f84a350fe2c
[ 245.682956] RSP: 002b:00007fffc13f5ac0 EFLAGS: 00000246 ORIG_RAX: 0000000000000000
[ 245.683337] RAX: ffffffffffffffda RBX: 000055b22c2b53c0 RCX: 00007f84a350fe2c
[ 245.683681] RDX: 0000000000000400 RSI: 000055b22c2b55f0 RDI: 0000000000000017
[ 245.683984] RBP: 00007f84a35ee540 R08: 0000000000000000 R09: 0000000000000001
[ 245.684291] R10: 0000000000001000 R11: 0000000000000246 R12: 00007fffc13f5bef
[ 245.684592] R13: 0000000000000d68 R14: 00007f84a35ed940 R15: 0000000000000d68
[ 245.685192] </TASK>
[ 260.654928] watchdog: BUG: soft lockup - CPU#0 stuck for 186s! [systemd:1]
[ 260.655464] Modules linked in: ext4 crc32c_generic crc16 mbcache jbd2 sr_mod cdrom serio_raw atkbd libps2 vivaldi_fmap ata_generic pata_acpi virtio_pci virtio_pci_legacy_dev virtio_pci_modern_dev ata_piix floppy i8042 serio virtio_rng e
[ 260.655592] Unloaded tainted modules: crc32c_intel():1
[ 260.656328] CPU: 0 PID: 1 Comm: systemd Tainted: G L 5.19.5-arch1-1 #1 bcfbb9540ef8053872a922808497e5b96b6f8fc8
[ 260.656595] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS Arch Linux 1.16.0-3-3 04/01/2014
[ 260.656838] RIP: 0010:___slab_alloc+0x555/0x6b0
[ 260.656994] Code: 73 08 e9 d7 00 00 00 48 89 fb 48 8b 03 48 85 c0 0f 84 b7 fe ff ff 48 63 40 58 4d 8b 8c c5 d8 00 00 00 48 89 c7 4d 85 c9 74 0c <eb> 3e 49 8b 41 08 49 39 45 10 72 4b 48 8d 7b 10 44 3b 63 18 73 c7
[ 260.657467] RSP: 0018:ffffbb3fc0013b58 EFLAGS: 00000282
[ 260.657611] RAX: 0000000000000000 RBX: ffff9cab6e9e41c0 RCX: 0000000000000000
[ 260.657799] RDX: 0000000000000000 RSI: ffff9cab41041280 RDI: 0000000000000000
[ 260.657980] RBP: 0000000000000dc0 R08: ffff9cab6e9e41c0 R09: ffff9cab41041280
[ 260.658246] R10: ffff9cab4623adc8 R11: ffffbb3fc0013c20 R12: 0000000000000002
[ 260.658246] R13: ffff9cab41042a00 R14: 00000000ffffffff R15: ffff9cab6da38140
[ 260.658796] FS: 00007f8020ae6a40(0000) GS:ffff9cab6da00000(0000) knlGS:0000000000000000
[ 260.658796] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 260.658796] CR2: 00007f654f796445 CR3: 000000002f494000 CR4: 00000000000006f0
[ 260.658796] Call Trace:
[ 260.658796] <TASK>
[ 260.658796] ? cpuset_css_alloc+0x2f/0x150
[ 260.658796] ? cgroup_addrm_files+0x320/0x370
[ 260.658796] ? cpuset_css_alloc+0x2f/0x150
[ 260.658796] kmem_cache_alloc_trace+0x29e/0x310
[ 260.658796] cpuset_css_alloc+0x2f/0x150
[ 260.658796] cgroup_apply_control_enable+0x120/0x340
[ 260.658796] cgroup_subtree_control_write+0x37b/0x400
[ 260.658796] kernfs_fop_write_iter+0x122/0x200
[ 260.658796] new_sync_write+0x13a/0x1c0
[ 260.658796] vfs_write+0x1ef/0x270
[ 260.658796] ksys_write+0x6f/0xf0
[ 260.658796] do_syscall_64+0x5f/0x90
[ 260.658796] ? vfs_write+0x178/0x270
[ 260.658796] ? vfs_write+0x178/0x270
[ 260.658796] ? ksys_write+0x6f/0xf0
[ 260.662060] ? syscall_exit_to_user_mode+0x1b/0x40
[ 260.662060] ? do_syscall_64+0x6b/0x90
[ 260.662060] ? do_syscall_64+0x6b/0x90
[ 260.662060] ? do_syscall_64+0x6b/0x90
[ 260.662060] entry_SYSCALL_64_after_hwframe+0x63/0xcd
[ 260.662060] RIP: 0033:0x7f8020f0fe94
[ 260.662060] Code: 15 f9 0e 0e 00 f7 d8 64 89 02 48 c7 c0 ff ff ff ff eb b7 0f 1f 00 f3 0f 1e fa 80 3d 8d 96 0e 00 00 74 13 b8 01 00 00 00 0f 05 <48> 3d 00 f0 ff ff 77 54 c3 0f 1f 00 48 83 ec 28 48 89 54 24 18 48
[ 260.662060] RSP: 002b:00007fff61974728 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 260.662060] RAX: ffffffffffffffda RBX: 0000000000000008 RCX: 00007f8020f0fe94
[ 260.662060] RDX: 0000000000000008 RSI: 000055ef79cdf1c0 RDI: 000000000000003f
[ 260.662060] RBP: 000055ef79cdf1c0 R08: fefefefefefefeff R09: ff736472746f622a
[ 260.662060] R10: 00007f8020ebfe00 R11: 0000000000000202 R12: 0000000000000008
[ 260.662060] R13: 000055ef79cd2130 R14: 0000000000000008 R15: 00007f8020fed940
[ 260.662060] </TASK>
[ 288.654930] watchdog: BUG: soft lockup - CPU#0 stuck for 212s! [systemd:1]
[ 288.655549] Modules linked in: ext4 crc32c_generic crc16 mbcache jbd2 sr_mod cdrom serio_raw atkbd libps2 vivaldi_fmap ata_generic pata_acpi virtio_pci virtio_pci_legacy_dev virtio_pci_modern_dev ata_piix floppy i8042 serio virtio_rng e
[ 288.655674] Unloaded tainted modules: crc32c_intel():1
[ 288.656553] CPU: 0 PID: 1 Comm: systemd Tainted: G L 5.19.5-arch1-1 #1 bcfbb9540ef8053872a922808497e5b96b6f8fc8
[ 288.657166] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS Arch Linux 1.16.0-3-3 04/01/2014
[ 288.657674] RIP: 0010:___slab_alloc+0x555/0x6b0
[ 288.657954] Code: 73 08 e9 d7 00 00 00 48 89 fb 48 8b 03 48 85 c0 0f 84 b7 fe ff ff 48 63 40 58 4d 8b 8c c5 d8 00 00 00 48 89 c7 4d 85 c9 74 0c <eb> 3e 49 8b 41 08 49 39 45 10 72 4b 48 8d 7b 10 44 3b 63 18 73 c7
[ 288.658954] RSP: 0018:ffffbb3fc0013b58 EFLAGS: 00000282
[ 288.658954] RAX: 0000000000000000 RBX: ffff9cab6e9e41c0 RCX: 0000000000000000
[ 288.658954] RDX: 0000000000000000 RSI: ffff9cab41041280 RDI: 0000000000000000
[ 288.658954] RBP: 0000000000000dc0 R08: ffff9cab6e9e41c0 R09: ffff9cab41041280
[ 288.658954] R10: ffff9cab4623adc8 R11: ffffbb3fc0013c20 R12: 0000000000000002
[ 288.658954] R13: ffff9cab41042a00 R14: 00000000ffffffff R15: ffff9cab6da38140
[ 288.658954] FS: 00007f8020ae6a40(0000) GS:ffff9cab6da00000(0000) knlGS:0000000000000000
[ 288.658954] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 288.658954] CR2: 00007f654f796445 CR3: 000000002f494000 CR4: 00000000000006f0
[ 288.658954] Call Trace:
[ 288.658954] <TASK>
[ 288.658954] ? cpuset_css_alloc+0x2f/0x150
[ 288.658954] ? cgroup_addrm_files+0x320/0x370
[ 288.658954] ? cpuset_css_alloc+0x2f/0x150
[ 288.658954] kmem_cache_alloc_trace+0x29e/0x310
[ 288.658954] cpuset_css_alloc+0x2f/0x150
[ 288.658954] cgroup_apply_control_enable+0x120/0x340
[ 288.658954] cgroup_subtree_control_write+0x37b/0x400
[ 288.665635] kernfs_fop_write_iter+0x122/0x200
[ 288.665635] new_sync_write+0x13a/0x1c0
[ 288.665635] vfs_write+0x1ef/0x270
[ 288.665635] ksys_write+0x6f/0xf0
[ 288.665635] do_syscall_64+0x5f/0x90
[ 288.665635] ? vfs_write+0x178/0x270
[ 288.665635] ? vfs_write+0x178/0x270
[ 288.665635] ? ksys_write+0x6f/0xf0
[ 288.665635] ? syscall_exit_to_user_mode+0x1b/0x40
[ 288.665635] ? do_syscall_64+0x6b/0x90
[ 288.665635] ? do_syscall_64+0x6b/0x90
[ 288.665635] ? do_syscall_64+0x6b/0x90
[ 288.665635] entry_SYSCALL_64_after_hwframe+0x63/0xcd
[ 288.665635] RIP: 0033:0x7f8020f0fe94
[ 288.665635] Code: 15 f9 0e 0e 00 f7 d8 64 89 02 48 c7 c0 ff ff ff ff eb b7 0f 1f 00 f3 0f 1e fa 80 3d 8d 96 0e 00 00 74 13 b8 01 00 00 00 0f 05 <48> 3d 00 f0 ff ff 77 54 c3 0f 1f 00 48 83 ec 28 48 89 54 24 18 48
[ 288.665635] RSP: 002b:00007fff61974728 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 288.665635] RAX: ffffffffffffffda RBX: 0000000000000008 RCX: 00007f8020f0fe94
[ 288.665635] RDX: 0000000000000008 RSI: 000055ef79cdf1c0 RDI: 000000000000003f
[ 288.665635] RBP: 000055ef79cdf1c0 R08: fefefefefefefeff R09: ff736472746f622a
[ 288.672812] R10: 00007f8020ebfe00 R11: 0000000000000202 R12: 0000000000000008
[ 288.672812] R13: 000055ef79cd2130 R14: 0000000000000008 R15: 00007f8020fed940
[ 288.672812] </TASK>
[ 302.215159] rcu: INFO: rcu_preempt detected stalls on CPUs/tasks:
[ 302.215633] (detected by 3, t=72007 jiffies, g=18269, q=166 ncpus=4)
[ 302.215941] rcu: All QSes seen, last rcu_preempt kthread activity 72007 (4294967876-4294895869), jiffies_till_next_fqs=2, root ->qsmask 0x0
[ 302.216702] rcu: rcu_preempt kthread starved for 72007 jiffies! g18269 f0x2 RCU_GP_WAIT_FQS(5) ->state=0x0 ->cpu=0
[ 302.217322] rcu: Unless rcu_preempt kthread gets sufficient CPU time, OOM is now expected behavior.
[ 302.217785] rcu: RCU grace-period kthread stack dump:
[ 302.218044] task:rcu_preempt state:R running task stack: 0 pid: 16 ppid: 2 flags:0x00004000
[ 302.218455] Call Trace:
[ 302.218455] <TASK>
[ 302.218455] ? rcu_gp_cleanup+0x3c0/0x3c0
[ 302.218455] __schedule+0x356/0x11a0
[ 302.218455] ? __mod_timer+0x289/0x3b0
[ 302.218455] ? rcu_gp_cleanup+0x3c0/0x3c0
[ 302.218455] schedule+0x5e/0xd0
[ 302.218455] schedule_timeout+0x98/0x150
[ 302.218455] ? __bpf_trace_tick_stop+0x10/0x10
[ 302.218455] rcu_gp_fqs_loop+0x113/0x3c0
[ 302.218455] rcu_gp_kthread+0xb0/0x170
[ 302.218455] kthread+0xde/0x110
[ 302.218455] ? kthread_complete_and_exit+0x20/0x20
[ 302.218455] ret_from_fork+0x22/0x30
[ 302.218455] </TASK>
[ 302.218455] rcu: Stack dump where RCU GP kthread last ran:
[ 302.218455] Sending NMI from CPU 3 to CPUs 0:
[ 302.223087] NMI backtrace for cpu 0
[ 302.223263] CPU: 0 PID: 1 Comm: systemd Tainted: G L 5.19.5-arch1-1 #1 bcfbb9540ef8053872a922808497e5b96b6f8fc8
[ 302.223290] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS Arch Linux 1.16.0-3-3 04/01/2014
[ 302.223299] RIP: 0010:asm_exc_int3+0x0/0x40
[ 302.223336] Code: e9 d5 0a 00 00 0f 1f 44 00 00 f3 0f 1e fa 0f 1f 00 fc 6a ff e8 a1 09 00 00 48 89 c4 48 89 e7 e8 c6 77 fd ff e9 b1 0a 00 00 90 <f3> 0f 1e fa 0f 1f 00 fc 6a ff f6 44 24 10 03 75 18 ff 74 24 28 ff
[ 302.223346] RSP: 0018:ffffbb3fc0013b28 EFLAGS: 00000082
[ 302.223363] RAX: 0000000000000000 RBX: ffff9cab6e9e41c0 RCX: 0000000000000000
[ 302.223371] RDX: 0000000000000000 RSI: ffff9cab41041280 RDI: 0000000000000000
[ 302.223377] RBP: 0000000000000dc0 R08: ffff9cab6e9e41c0 R09: ffff9cab41041280
[ 302.223383] R10: ffff9cab4623adc8 R11: ffffbb3fc0013c20 R12: 0000000000000002
[ 302.223388] R13: ffff9cab41042a00 R14: 00000000ffffffff R15: ffff9cab6da38140
[ 302.223397] FS: 00007f8020ae6a40(0000) GS:ffff9cab6da00000(0000) knlGS:0000000000000000
[ 302.223405] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 302.223411] CR2: 00007f654f796445 CR3: 000000002f494000 CR4: 00000000000006f0
[ 302.223424] Call Trace:
[ 302.223443] <TASK>
[ 302.223447] RIP: 0010:___slab_alloc+0x556/0x6b0
[ 302.223463] Code: 08 e9 d7 00 00 00 48 89 fb 48 8b 03 48 85 c0 0f 84 b7 fe ff ff 48 63 40 58 4d 8b 8c c5 d8 00 00 00 48 89 c7 4d 85 c9 74 0c eb <3e> 49 8b 41 08 49 39 45 10 72 4b 48 8d 7b 10 44 3b 63 18 73 c7 31
[ 302.223471] RSP: 0018:ffffbb3fc0013b58 EFLAGS: 00000282
[ 302.223516] ? ___slab_alloc+0x50d/0x6b0
[ 302.223530] ? cpuset_css_alloc+0x2f/0x150
[ 302.223541] ? cgroup_addrm_files+0x320/0x370
[ 302.223561] ? cpuset_css_alloc+0x2f/0x150
[ 302.223568] kmem_cache_alloc_trace+0x29e/0x310
[ 302.223587] cpuset_css_alloc+0x2f/0x150
[ 302.223597] cgroup_apply_control_enable+0x120/0x340
[ 302.223615] cgroup_subtree_control_write+0x37b/0x400
[ 302.223641] kernfs_fop_write_iter+0x122/0x200
[ 302.223661] new_sync_write+0x13a/0x1c0
[ 302.223687] vfs_write+0x1ef/0x270
[ 302.223700] ksys_write+0x6f/0xf0
[ 302.223713] do_syscall_64+0x5f/0x90
[ 302.223734] ? vfs_write+0x178/0x270
[ 302.223743] ? vfs_write+0x178/0x270
[ 302.223753] ? ksys_write+0x6f/0xf0
[ 302.223764] ? syscall_exit_to_user_mode+0x1b/0x40
[ 302.223773] ? do_syscall_64+0x6b/0x90
[ 302.223784] ? do_syscall_64+0x6b/0x90
[ 302.223794] ? do_syscall_64+0x6b/0x90
[ 302.223807] entry_SYSCALL_64_after_hwframe+0x63/0xcd
[ 302.223820] RIP: 0033:0x7f8020f0fe94
[ 302.223848] Code: 15 f9 0e 0e 00 f7 d8 64 89 02 48 c7 c0 ff ff ff ff eb b7 0f 1f 00 f3 0f 1e fa 80 3d 8d 96 0e 00 00 74 13 b8 01 00 00 00 0f 05 <48> 3d 00 f0 ff ff 77 54 c3 0f 1f 00 48 83 ec 28 48 89 54 24 18 48
[ 302.223855] RSP: 002b:00007fff61974728 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 302.223867] RAX: ffffffffffffffda RBX: 0000000000000008 RCX: 00007f8020f0fe94
[ 302.223874] RDX: 0000000000000008 RSI: 000055ef79cdf1c0 RDI: 000000000000003f
[ 302.223879] RBP: 000055ef79cdf1c0 R08: fefefefefefefeff R09: ff736472746f622a
[ 302.223884] R10: 00007f8020ebfe00 R11: 0000000000000202 R12: 0000000000000008
[ 302.223890] R13: 000055ef79cd2130 R14: 0000000000000008 R15: 00007f8020fed940
[ 302.223911] </TASK>
```

```
[ 491.433151] INFO: task kworker/u8:0:8 blocked for more than 368 seconds.
[ 491.433671] Not tainted 5.19.5-arch1-1 #1
[ 491.433965] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 491.434492] task:kworker/u8:0 state:D stack: 0 pid: 8 ppid: 2 flags:0x00004000
[ 491.435131] Workqueue: writeback wb_workfn (flush-8:0)
[ 491.435522] Call Trace:
[ 491.435734] <TASK>
[ 491.435912] __schedule+0x356/0x11a0
[ 491.436236] ? wb_over_bg_thresh+0x26c/0x2a0
[ 491.436751] schedule+0x5e/0xd0
[ 491.436990] inode_sleep_on_writeback+0xbe/0xd0
[ 491.437374] ? cpuacct_percpu_seq_show+0x20/0x20
[ 491.437749] wb_writeback+0x11f/0x2f0
[ 491.438047] wb_workfn+0x2c9/0x520
[ 491.438300] process_one_work+0x1c7/0x380
[ 491.438605] worker_thread+0x51/0x390
[ 491.438904] ? rescuer_thread+0x3b0/0x3b0
[ 491.439212] kthread+0xde/0x110
[ 491.439471] ? kthread_complete_and_exit+0x20/0x20
[ 491.439989] ret_from_fork+0x22/0x30
[ 491.440281] </TASK>
[ 491.440504] INFO: task kworker/u8:2:45 blocked for more than 368 seconds.
[ 491.440960] Not tainted 5.19.5-arch1-1 #1
[ 491.441279] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 491.441875] task:kworker/u8:2 state:D stack: 0 pid: 45 ppid: 2 flags:0x00004000
[ 491.442454] Workqueue: writeback wb_workfn (flush-8:0)
[ 491.442855] Call Trace:
[ 491.453066] <TASK>
[ 491.453251] __schedule+0x356/0x11a0
[ 491.453494] ? kmem_cache_alloc+0x16a/0x300
[ 491.453830] schedule+0x5e/0xd0
[ 491.454090] io_schedule+0x46/0x70
[ 491.454307] blk_mq_get_tag+0x11e/0x2a0
[ 491.454604] ? cpuacct_percpu_seq_show+0x20/0x20
[ 491.454965] __blk_mq_alloc_requests+0x184/0x2e0
[ 491.455300] blk_mq_submit_bio+0x2b4/0x5b0
[ 491.455653] __submit_bio+0xf5/0x180
[ 491.455913] submit_bio_noacct_nocheck+0x20e/0x2c0
[ 491.456227] ext4_io_submit+0x24/0x40 [ext4 469a96554db3024f50e2546d8f81b9874cc9bfc2]
[ 491.457089] ext4_writepages+0x4b6/0x1200 [ext4 469a96554db3024f50e2546d8f81b9874cc9bfc2]
[ 491.457870] do_writepages+0xc4/0x1d0
[ 491.458172] ? wb_get_create+0x62/0x6a0
[ 491.458449] __writeback_single_inode+0x3d/0x360
[ 491.458818] writeback_sb_inodes+0x1ed/0x4a0
[ 491.459154] __writeback_inodes_wb+0x4c/0xf0
[ 491.459484] wb_writeback+0x204/0x2f0
[ 491.459964] wb_workfn+0x36f/0x520
[ 491.460268] process_one_work+0x1c7/0x380
[ 491.460544] worker_thread+0x51/0x390
[ 491.460816] ? rescuer_thread+0x3b0/0x3b0
[ 491.461083] kthread+0xde/0x110
[ 491.461341] ? kthread_complete_and_exit+0x20/0x20
[ 491.461757] ret_from_fork+0x22/0x30
[ 491.462047] </TASK>
[ 491.462305] INFO: task kworker/u8:5:894 blocked for more than 368 seconds.
[ 491.462820] Not tainted 5.19.5-arch1-1 #1
[ 491.473057] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 491.473639] task:kworker/u8:5 state:D stack: 0 pid: 894 ppid: 2 flags:0x00004000
[ 491.474228] Workqueue: writeback wb_workfn (flush-8:0)
[ 491.474612] Call Trace:
[ 491.474825] <TASK>
[ 491.474981] __schedule+0x356/0x11a0
[ 491.475257] ? rwb_trace_step+0x90/0x90
[ 491.475566] ? wbt_done+0xc0/0xc0
[ 491.475805] schedule+0x5e/0xd0
[ 491.476036] io_schedule+0x46/0x70
[ 491.476444] rq_qos_wait+0xc0/0x140
[ 491.476740] ? karma_partition+0x2a0/0x2a0
[ 491.477071] ? rwb_trace_step+0x90/0x90
[ 491.477390] wbt_wait+0xa9/0x110
[ 491.477626] __rq_qos_throttle+0x27/0x40
[ 491.477884] blk_mq_submit_bio+0x289/0x5b0
[ 491.478176] __submit_bio+0xf5/0x180
[ 491.478483] submit_bio_noacct_nocheck+0x20e/0x2c0
[ 491.478850] submit_bh_wbc+0x12b/0x150
[ 491.479136] __block_write_full_page+0x1fe/0x500
[ 491.479484] ? bh_uptodate_or_lock+0x80/0x80
[ 491.480016] ? blkdev_llseek+0x60/0x60
[ 491.480308] __writepage+0x1d/0x70
[ 491.480542] write_cache_pages+0x187/0x4c0
[ 491.480884] ? dirty_background_bytes_handler+0x30/0x30
[ 491.481255] generic_writepages+0x7f/0xb0
[ 491.481566] do_writepages+0xc4/0x1d0
[ 491.481857] __writeback_single_inode+0x3d/0x360
[ 491.482191] writeback_sb_inodes+0x1ed/0x4a0
[ 491.482560] __writeback_inodes_wb+0x4c/0xf0
[ 491.482859] wb_writeback+0x204/0x2f0
[ 491.493105] wb_workfn+0x36f/0x520
[ 491.493339] ? check_preempt_curr+0x61/0x70
[ 491.493670] ? ttwu_do_wakeup+0x17/0x170
[ 491.493970] process_one_work+0x1c7/0x380
[ 491.494261] worker_thread+0x51/0x390
[ 491.494571] ? rescuer_thread+0x3b0/0x3b0
[ 491.494856] kthread+0xde/0x110
[ 491.495092] ? kthread_complete_and_exit+0x20/0x20
[ 491.495460] ret_from_fork+0x22/0x30
[ 491.495779] </TASK>
[ 527.728135] systemd[1]: systemd-udevd.service: Got notification message from PID 226 (WATCHDOG=1)
[ 558.726681] systemd[1]: systemd-journald.service: State 'final-sigterm' timed out. Killing.
[ 558.727927] systemd[1]: systemd-journald.service: Killing process 207 (8) with signal SIGKILL.
[ 558.729607] systemd[1]: systemd-journald.service: Changed final-sigterm -> final-sigkill
[ 558.731743] systemd[1]: Sent message type=signal sender=n/a destination=n/a path=/org/freedesktop/systemd1/unit/systemd_2djournald_2eservice interface=org.freedesktop.DBus.Properties member=PropertiesChanged cookie=1276 reply_cookie=0 a
[ 558.734554] systemd[1]: Sent message type=signal sender=n/a destination=n/a path=/org/freedesktop/systemd1/unit/systemd_2djournald_2eservice interface=org.freedesktop.DBus.Properties member=PropertiesChanged cookie=1277 reply_cookie=0 a
[ 558.743693] systemd[1]: systemd-machined.service: Got notification message from PID 283 (WATCHDOG=1)
[ 558.744985] systemd[1]: systemd-logind.service: Got notification message from PID 244 (WATCHDOG=1)
```

```
[ 1772.741388] watchdog: BUG: soft lockup - CPU#1 stuck for 1538s! [kworker/u8:2:45]
[ 1772.741919] Modules linked in: ext4 crc32c_generic crc16 mbcache jbd2 sr_mod cdrom serio_raw atkbd libps2 vivaldi_fmap ata_generic pata_acpi virtio_pci virtio_pci_legacy_dev virtio_pci_modern_dev ata_piix floppy i8042 serio virtio_rng e
[ 1772.742035] Unloaded tainted modules: crc32c_intel():1
[ 1772.744000] CPU: 1 PID: 45 Comm: kworker/u8:2 Tainted: G L 5.19.5-arch1-1 #1 bcfbb9540ef8053872a922808497e5b96b6f8fc8
[ 1772.744712] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS Arch Linux 1.16.0-3-3 04/01/2014
[ 1772.744712] Workqueue: writeback wb_workfn (flush-8:0)
[ 1772.744712] RIP: 0010:___slab_alloc+0x555/0x6b0
[ 1772.744712] Code: 73 08 e9 d7 00 00 00 48 89 fb 48 8b 03 48 85 c0 0f 84 b7 fe ff ff 48 63 40 58 4d 8b 8c c5 d8 00 00 00 48 89 c7 4d 85 c9 74 0c <eb> 3e 49 8b 41 08 49 39 45 10 72 4b 48 8d 7b 10 44 3b 63 18 73 c7
[ 1772.744712] RSP: 0018:ffffc09540197820 EFLAGS: 00000282
[ 1772.744712] RAX: 0000000000000000 RBX: ffff9ffe2e9e41c0 RCX: 0000000000000000
[ 1772.744712] RDX: 0000000000000000 RSI: ffff9ffe019a7a40 RDI: 0000000000000000
[ 1772.744712] RBP: 0000000000092800 R08: ffff9ffe2e9e41c0 R09: ffff9ffe019a7a40
[ 1772.744712] R10: 0000000000039408 R11: ffff9ffe2e9e26c0 R12: 0000000000000002
[ 1772.744712] R13: ffff9ffe019a6500 R14: 00000000ffffffff R15: ffff9ffe2dabb590
[ 1772.744712] FS: 0000000000000000(0000) GS:ffff9ffe2da80000(0000) knlGS:0000000000000000
[ 1772.744712] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 1772.744712] CR2: 000055963d3a67e8 CR3: 00000000038da000 CR4: 00000000000006e0
[ 1772.744712] Call Trace:
[ 1772.744712] <TASK>
[ 1772.744712] ? mempool_alloc+0x89/0x1b0
[ 1772.744712] ? ll_back_merge_fn+0x19c/0x210
[ 1772.744712] ? bio_attempt_back_merge+0x7f/0xe0
[ 1772.744712] ? mempool_alloc+0x89/0x1b0
[ 1772.744712] kmem_cache_alloc+0x29c/0x300
[ 1772.744712] mempool_alloc+0x89/0x1b0
[ 1772.744712] bio_alloc_bioset+0x1f1/0x4c0
[ 1772.744712] submit_bh_wbc+0x9c/0x150
[ 1772.744712] __block_write_full_page+0x1fe/0x500
[ 1772.744712] ? bh_uptodate_or_lock+0x80/0x80
[ 1772.744712] ? blkdev_llseek+0x60/0x60
[ 1772.744712] __writepage+0x1d/0x70
[ 1772.744712] write_cache_pages+0x187/0x4c0
[ 1772.744712] ? dirty_background_bytes_handler+0x30/0x30
[ 1772.744712] generic_writepages+0x7f/0xb0
[ 1772.744712] do_writepages+0xc4/0x1d0
[ 1772.744712] __writeback_single_inode+0x3d/0x360
[ 1772.744712] writeback_sb_inodes+0x1ed/0x4a0
[ 1772.744712] __writeback_inodes_wb+0x4c/0xf0
[ 1772.744712] wb_writeback+0x204/0x2f0
[ 1772.744712] wb_workfn+0x36f/0x520
[ 1772.744712] process_one_work+0x1c7/0x380
[ 1772.744712] worker_thread+0x51/0x390
[ 1772.744712] ? rescuer_thread+0x3b0/0x3b0
[ 1772.744712] kthread+0xde/0x110
[ 1772.744712] ? kthread_complete_and_exit+0x20/0x20
[ 1772.744712] ret_from_fork+0x22/0x30
[ 1772.744712] </TASK>
[ 1784.644712] watchdog: BUG: soft lockup - CPU#0 stuck for 1133s! [6:226]
[ 1784.645228] Modules linked in: ext4 crc32c_generic crc16 mbcache jbd2 sr_mod cdrom serio_raw atkbd libps2 vivaldi_fmap ata_generic pata_acpi virtio_pci virtio_pci_legacy_dev virtio_pci_modern_dev ata_piix floppy i8042 serio virtio_rng e
[ 1784.645349] Unloaded tainted modules: crc32c_intel():1
[ 1784.647541] CPU: 0 PID: 226 Comm: 6 Tainted: G L 5.19.5-arch1-1 #1 bcfbb9540ef8053872a922808497e5b96b6f8fc8
[ 1784.648021] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS Arch Linux 1.16.0-3-3 04/01/2014
[ 1784.648021] RIP: 0010:___slab_alloc+0x555/0x6b0
[ 1784.648021] Code: 73 08 e9 d7 00 00 00 48 89 fb 48 8b 03 48 85 c0 0f 84 b7 fe ff ff 48 63 40 58 4d 8b 8c c5 d8 00 00 00 48 89 c7 4d 85 c9 74 0c <eb> 3e 49 8b 41 08 49 39 45 10 72 4b 48 8d 7b 10 44 3b 63 18 73 c7
[ 1784.648021] RSP: 0018:ffffc0954033bc90 EFLAGS: 00000282
[ 1784.648021] RAX: 0000000000000000 RBX: ffff9ffe2e9e41c0 RCX: 0000000000000000
[ 1784.648021] RDX: 0000000000000000 RSI: ffff9ffe011c5d40 RDI: 0000000000000000
[ 1784.648021] RBP: 0000000000000dc0 R08: ffff9ffe2e9e41c0 R09: ffff9ffe011c5d40
[ 1784.648021] R10: 0000000000000000 R11: 0000000000000001 R12: 0000000000000002
[ 1784.648021] R13: ffff9ffe011c9800 R14: 00000000ffffffff R15: ffff9ffe2da39820
[ 1784.648021] FS: 00007f369ad0d200(0000) GS:ffff9ffe2da00000(0000) knlGS:0000000000000000
[ 1784.648021] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 1784.648021] CR2: 00005585875d1880 CR3: 000000002f344000 CR4: 00000000000006f0
[ 1784.648021] Call Trace:
[ 1784.648021] <TASK>
[ 1784.648021] ? __alloc_file+0x23/0xd0
[ 1784.648021] ? memcg_list_lru_alloc+0xaf/0x3d0
[ 1784.648021] ? __alloc_file+0x23/0xd0
[ 1784.648021] kmem_cache_alloc+0x29c/0x300
[ 1784.648021] __alloc_file+0x23/0xd0
[ 1784.648021] alloc_empty_file+0x41/0xc0
[ 1784.648021] alloc_file+0x2b/0x110
[ 1784.648021] alloc_file_pseudo+0xaf/0x120
[ 1784.648021] sock_alloc_file+0x3d/0xa0
[ 1784.648021] __sys_socket+0xbf/0x100
[ 1784.648021] __x64_sys_socket+0x17/0x20
[ 1784.648021] do_syscall_64+0x5f/0x90
[ 1784.648021] ? ktime_get_ts64+0x4a/0xe0
[ 1784.648021] ? syscall_exit_to_user_mode+0x1b/0x40
[ 1784.648021] ? do_syscall_64+0x6b/0x90
[ 1784.648021] ? do_syscall_64+0x6b/0x90
[ 1784.648021] entry_SYSCALL_64_after_hwframe+0x63/0xcd
[ 1784.648021] RIP: 0033:0x7f369b6cbb8b
[ 1784.648021] Code: 73 01 c3 48 8b 0d fd e1 0c 00 f7 d8 64 89 01 48 83 c8 ff c3 66 2e 0f 1f 84 00 00 00 00 00 90 f3 0f 1e fa b8 29 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d cd e1 0c 00 f7 d8 64 89 01 48
[ 1784.648021] RSP: 002b:00007ffd7827b0c8 EFLAGS: 00000202 ORIG_RAX: 0000000000000029
[ 1784.648021] RAX: ffffffffffffffda RBX: 00007f369baca7b8 RCX: 00007f369b6cbb8b
[ 1784.648021] RDX: 0000000000000000 RSI: 0000000000080002 RDI: 0000000000000001
[ 1784.648021] RBP: 00007ffd7827b1f0 R08: 0000000000000010 R09: 00000000ffffffff
[ 1784.648021] R10: 00007ffd782cd080 R11: 0000000000000202 R12: 0000000000000000
[ 1784.648021] R13: 0000000000000016 R14: ffffffffffffffff R15: 00007f369bb66000
[ 1784.648021] </TASK>
[ 1784.831433] watchdog: BUG: soft lockup - CPU#2 stuck for 1554s! [systemd-nspawn:1353]
[ 1784.832065] Modules linked in: ext4 crc32c_generic crc16 mbcache jbd2 sr_mod cdrom serio_raw atkbd libps2 vivaldi_fmap ata_generic pata_acpi virtio_pci virtio_pci_legacy_dev virtio_pci_modern_dev ata_piix floppy i8042 serio virtio_rng e
[ 1784.832176] Unloaded tainted modules: crc32c_intel():1
[ 1784.834219] CPU: 2 PID: 1353 Comm: systemd-nspawn Tainted: G L 5.19.5-arch1-1 #1 bcfbb9540ef8053872a922808497e5b96b6f8fc8
[ 1784.834647] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS Arch Linux 1.16.0-3-3 04/01/2014
[ 1784.834647] RIP: 0010:___slab_alloc+0x555/0x6b0
[ 1784.834647] Code: 73 08 e9 d7 00 00 00 48 89 fb 48 8b 03 48 85 c0 0f 84 b7 fe ff ff 48 63 40 58 4d 8b 8c c5 d8 00 00 00 48 89 c7 4d 85 c9 74 0c <eb> 3e 49 8b 41 08 49 39 45 10 72 4b 48 8d 7b 10 44 3b 63 18 73 c7
[ 1784.834647] RSP: 0018:ffffc0954056fc18 EFLAGS: 00000282
[ 1784.834647] RAX: 0000000000000000 RBX: ffff9ffe2e9e41c0 RCX: 0000000000000000
[ 1784.834647] RDX: 0000000000000000 RSI: ffff9ffe01041280 RDI: 0000000000000000
[ 1784.834647] RBP: 0000000000000dc0 R08: ffff9ffe2e9e41c0 R09: ffff9ffe01041280
[ 1784.834647] R10: 000000000000011b R11: 0000000000000001 R12: 0000000000000002
[ 1784.834647] R13: ffff9ffe01042a00 R14: 00000000ffffffff R15: ffff9ffe2db38140
[ 1784.834647] FS: 00007f7545bb7fc0(0000) GS:ffff9ffe2db00000(0000) knlGS:0000000000000000
[ 1784.834647] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 1784.834647] CR2: 00007f75466a0dc9 CR3: 0000000002a72000 CR4: 00000000000006e0
[ 1784.834647] Call Trace:
[ 1784.834647] <TASK>
[ 1784.834647] ? blkcg_css_alloc+0x3b/0x1c0
[ 1784.834647] ? css_next_descendant_post+0x23/0x80
[ 1784.834647] ? cgroup_apply_control_disable+0x5e/0x1a0
[ 1784.834647] ? blkcg_css_alloc+0x3b/0x1c0
[ 1784.834647] kmem_cache_alloc_trace+0x29e/0x310
[ 1784.834647] blkcg_css_alloc+0x3b/0x1c0
[ 1784.834647] cgroup_apply_control_enable+0x120/0x340
[ 1784.834647] cgroup_subtree_control_write+0x37b/0x400
[ 1784.834647] kernfs_fop_write_iter+0x122/0x200
[ 1784.834647] new_sync_write+0x13a/0x1c0
[ 1784.834647] vfs_write+0x1ef/0x270
[ 1784.834647] ksys_write+0x6f/0xf0
[ 1784.834647] do_syscall_64+0x5f/0x90
[ 1784.834647] ? __irq_exit_rcu+0x4b/0xf0
[ 1784.834647] entry_SYSCALL_64_after_hwframe+0x63/0xcd
[ 1784.834647] RIP: 0033:0x7f754650fe94
[ 1784.834647] Code: 15 f9 0e 0e 00 f7 d8 64 89 02 48 c7 c0 ff ff ff ff eb b7 0f 1f 00 f3 0f 1e fa 80 3d 8d 96 0e 00 00 74 13 b8 01 00 00 00 0f 05 <48> 3d 00 f0 ff ff 77 54 c3 0f 1f 00 48 83 ec 28 48 89 54 24 18 48
[ 1784.834647] RSP: 002b:00007fff691b04b8 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[ 1784.834647] RAX: ffffffffffffffda RBX: 0000000000000004 RCX: 00007f754650fe94
[ 1784.834647] RDX: 0000000000000004 RSI: 00005585875e1b00 RDI: 0000000000000011
[ 1784.834647] RBP: 00005585875e1b00 R08: fefefefefefefeff R09: ff736471ff6e682a
[ 1784.834647] R10: 00007f75465f1c80 R11: 0000000000000202 R12: 0000000000000004
[ 1784.834647] R13: 00005585875d05e0 R14: 0000000000000004 R15: 00007f75465ed940
[ 1784.834647] </TASK>
[ 1796.914743] watchdog: BUG: soft lockup - CPU#3 stuck for 1565s! [8:207]
[ 1796.915244] Modules linked in: ext4 crc32c_generic crc16 mbcache jbd2 sr_mod cdrom serio_raw atkbd libps2 vivaldi_fmap ata_generic pata_acpi virtio_pci virtio_pci_legacy_dev virtio_pci_modern_dev ata_piix floppy i8042 serio virtio_rng e
[ 1796.915354] Unloaded tainted modules: crc32c_intel():1
[ 1796.917193] CPU: 3 PID: 207 Comm: 8 Tainted: G L 5.19.5-arch1-1 #1 bcfbb9540ef8053872a922808497e5b96b6f8fc8
[ 1796.917969] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS Arch Linux 1.16.0-3-3 04/01/2014
[ 1796.917969] RIP: 0010:___slab_alloc+0x555/0x6b0
[ 1796.917969] Code: 73 08 e9 d7 00 00 00 48 89 fb 48 8b 03 48 85 c0 0f 84 b7 fe ff ff 48 63 40 58 4d 8b 8c c5 d8 00 00 00 48 89 c7 4d 85 c9 74 0c <eb> 3e 49 8b 41 08 49 39 45 10 72 4b 48 8d 7b 10 44 3b 63 18 73 c7
[ 1796.917969] RSP: 0018:ffffc09540553b70 EFLAGS: 00000282
[ 1796.917969] RAX: 0000000000000000 RBX: ffff9ffe2e9e41c0 RCX: 0000000000000000
[ 1796.917969] RDX: 0000000000000000 RSI: ffff9ffe011c5d40 RDI: 0000000000000000
[ 1796.917969] RBP: 0000000000000dc0 R08: ffff9ffe2e9e41c0 R09: ffff9ffe011c5d40
[ 1796.917969] R10: 0000000000000000 R11: 0000000000000001 R12: 0000000000000002
[ 1796.917969] R13: ffff9ffe011c9800 R14: 00000000ffffffff R15: ffff9ffe2dbb9820
[ 1796.917969] FS: 00007f0142b30fc0(0000) GS:ffff9ffe2db80000(0000) knlGS:0000000000000000
[ 1796.917969] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 1796.917969] CR2: 00007f0138c17000 CR3: 000000002f3fa000 CR4: 00000000000006e0
[ 1796.917969] Call Trace:
[ 1796.917969] <TASK>
[ 1796.917969] ? __alloc_file+0x23/0xd0
[ 1796.917969] ? refill_stock+0x1a/0x30
[ 1796.917969] ? __alloc_file+0x23/0xd0
[ 1796.917969] kmem_cache_alloc+0x29c/0x300
[ 1796.917969] __alloc_file+0x23/0xd0
[ 1796.917969] alloc_empty_file+0x41/0xc0
[ 1796.917969] path_openat+0x4a/0x1270
[ 1796.917969] ? ___sys_recvmsg+0xbd/0x130
[ 1796.917969] do_filp_open+0xb3/0x160
[ 1796.917969] do_sys_openat2+0xaf/0x170
[ 1796.917969] __x64_sys_openat+0x6e/0xa0
[ 1796.917969] do_syscall_64+0x5f/0x90
[ 1796.917969] ? syscall_exit_to_user_mode+0x1b/0x40
[ 1796.917969] ? do_syscall_64+0x6b/0x90
[ 1796.917969] ? do_syscall_64+0x6b/0x90
[ 1796.917969] ? do_syscall_64+0x6b/0x90
[ 1796.917969] ? syscall_exit_to_user_mode+0x1b/0x40
[ 1796.917969] ? do_syscall_64+0x6b/0x90
[ 1796.917969] entry_SYSCALL_64_after_hwframe+0x63/0xcd
[ 1796.917969] RIP: 0033:0x7f014270fd50
[ 1796.917969] Code: 89 7c 24 18 44 89 54 24 0c e8 bc be f8 ff 44 8b 54 24 0c 8b 54 24 1c 41 89 c0 48 8b 74 24 10 8b 7c 24 18 b8 01 01 00 00 0f 05 <48> 3d 00 f0 ff ff 77 38 44 89 c7 89 44 24 0c e8 0c bf f8 ff 8b 44
[ 1796.917969] RSP: 002b:00007ffd2afefcd0 EFLAGS: 00000293 ORIG_RAX: 0000000000000101
[ 1796.917969] RAX: ffffffffffffffda RBX: 0000556a345c4410 RCX: 00007f014270fd50
[ 1796.917969] RDX: 0000000000080802 RSI: 0000556a345c49c0 RDI: 00000000ffffff9c
[ 1796.917969] RBP: 00007ffd2afefd90 R08: 0000000000000000 R09: ffffffffffffffff
[ 1796.917969] R10: 0000000000000000 R11: 0000000000000293 R12: 0000000000080802
[ 1796.917969] R13: 00000000ffffff9c R14: 0000556a345c49c0 R15: 00000000000001a0
[ 1796.917969] </TASK>
```

Also, in one instance, the machine died with:
```
[ 1413.878040] BUG: kernel NULL pointer dereference, address: 0000000000000000
[ 1413.879468] #PF: supervisor instruction fetch in kernel mode
[ 1413.880495] #PF: error_code(0x0010) - not-present page
[ 1413.881421] PGD 0 P4D 0
[ 1413.881884] Oops: 0010 [#1] PREEMPT SMP PTI
[ 1413.882629] CPU: 64 PID: 67693 Comm: qemu-system-x86 Tainted: G W 5.19.5-arch1-1 #1 bcfbb9540ef8053872a922808497e5b96b6f8fc8
[ 1413.884738] Hardware name: Red Hat KVM, BIOS 1.16.0-1.module_el8.7.0+1140+ff0772f9 04/01/2014
[ 1413.886181] RIP: 0010:0x0
[ 1413.886652] Code: Unable to access opcode bytes at RIP 0xffffffffffffffd6.
[ 1413.887818] RSP: 0018:ffffb29bc1004ee8 EFLAGS: 00010202
[ 1413.888736] RAX: 0000000000000000 RBX: 0000000000000001 RCX: ffff8c57cbc33c78
[ 1413.889957] RDX: 0000000000000000 RSI: ffffb29bc1004ef8 RDI: ffff8c2c45f6edc0
[ 1413.891174] RBP: ffff8c57cbc33b40 R08: 0000000000000000 R09: 0000000000000100
[ 1413.892417] R10: 0000000000000200 R11: ffffffff8aa060c0 R12: ffff8c2dd7ef8000
[ 1413.893634] R13: 0000000000000000 R14: 000000000000000a R15: 0000000000000000
[ 1413.894851] FS: 00007f4819ffb6c0(0000) GS:ffff8c57cbc00000(0000) knlGS:0000000000000000
[ 1413.896224] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 1413.897198] CR2: ffffffffffffffd6 CR3: 00000002a9a94002 CR4: 0000000000770ee0
[ 1413.898422] PKRU: 55555554
[ 1413.898890] Call Trace:
[ 1413.899338] <IRQ>
[ 1413.899693] rcu_do_batch+0x199/0x4e0
[ 1413.900336] rcu_core+0x1ae/0x4d0
[ 1413.900912] __do_softirq+0xd0/0x2c9
[ 1413.901544] __irq_exit_rcu+0xbb/0xf0
[ 1413.902186] sysvec_apic_timer_interrupt+0x72/0x90
[ 1413.903016] </IRQ>
[ 1413.903400] <TASK>
[ 1413.903772] asm_sysvec_apic_timer_interrupt+0x1a/0x20
[ 1413.904653] RIP: 0010:syscall_enter_from_user_mode+0x16/0x60
[ 1413.905631] Code: 00 65 8b 05 3c 1e 65 76 83 f8 01 75 e6 eb db 0f 1f 44 00 00 66 0f 1f 00 55 48 89 fd 53 48 89 f3 66 90 66 90 fb 0f 1f 44 00 00 <65> 48 8b 04 25 c0 0b 02 00 48 8b 70 08 40 f6 c6 3f 75 1c 48 89 d8
[ 1413.908745] RSP: 0018:ffffb29bc40f3ea8 EFLAGS: 00000286
[ 1413.909644] RAX: ffffb29bc40f3ec0 RBX: 00000000000000ca RCX: 0000000000000048
[ 1413.910850] RDX: 0000000000000000 RSI: 00000000000000ca RDI: ffffb29bc40f3f58
[ 1413.912067] RBP: ffffb29bc40f3f58 R08: 0000000000000000 R09: 0000000000000000
[ 1413.913285] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
[ 1413.914492] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
[ 1413.915723] do_syscall_64+0x3a/0x90
[ 1413.916352] ? syscall_exit_to_user_mode+0x1b/0x40
[ 1413.917165] ? do_syscall_64+0x6b/0x90
[ 1413.917822] ? syscall_exit_to_user_mode+0x1b/0x40
[ 1413.918664] ? do_syscall_64+0x6b/0x90
[ 1413.919318] ? do_syscall_64+0x6b/0x90
[ 1413.919960] ? do_syscall_64+0x6b/0x90
[ 1413.920609] entry_SYSCALL_64_after_hwframe+0x63/0xcd
[ 1413.921483] RIP: 0033:0x7f49fcc315e7
[ 1413.922106] Code: 81 00 00 00 b8 ca 00 00 00 0f 05 c3 0f 1f 80 00 00 00 00 f3 0f 1e fa 40 80 f6 81 45 31 d2 ba 01 00 00 00 b8 ca 00 00 00 0f 05 <c3> 0f 1f 84 00 00 00 00 00 f3 0f 1e fa 48 8b 05 7d 49 15 00 48 89
[ 1413.925240] RSP: 002b:00007f4819ffa5f8 EFLAGS: 00000246 ORIG_RAX: 00000000000000ca
[ 1413.926524] RAX: ffffffffffffffda RBX: 000055aad47ae340 RCX: 00007f49fcc315e7
[ 1413.927741] RDX: 0000000000000001 RSI: 0000000000000081 RDI: 000055aad47ae340
[ 1413.928966] RBP: 0000000000010000 R08: ffffbfce02ebb000 R09: 00007f480817c6c8
[ 1413.930177] R10: 0000000000000000 R11: 0000000000000246 R12: 000055aad52936b0
[ 1413.931399] R13: 0000000000000000 R14: 00007f4819ffa790 R15: 00007f48197fb000
[ 1413.932625] </TASK>
[ 1413.933023] Modules linked in: sit bonding tls veth psample sch_tbf sch_sfq sch_sfb sch_qfq sch_netem sch_ingress sch_htb sch_hhf sch_gred sch_fq_pie sch_pie sch_fq sch_ets sch_drr sch_codel sch_cake tcp_dctcp l2tp_ip l2tp_eth ifb fou xfrm_interface xfrm6_tunnel tunnel4 tunnel6 ip_gre ip_tunnel gre wireguard curve25519_x86_64 libchacha20poly1305 chacha_x86_64 poly1305_x86_64 libcurve25519_generic libchacha vxcan vcan vrf ipvtap tap ipvlan batman_adv bridge bareudp dummy tun xt_nat xt_addrtype vfat xt_tcpudp fat xt_MASQUERADE iptable_nat nft_fib_ipv6 loop nft_masq nft_nat nft_fib_ipv4 nft_fib nft_chain_nat nf_nat nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 nf_tables libcrc32c nfnetlink macsec l2tp_netlink l2tp_core ip6_udp_tunnel udp_tunnel rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver nfs lockd grace sunrpc fscache netfs intel_rapl_msr intel_rapl_common intel_uncore_frequency_common isst_if_common nfit kvm_intel kvm irqbypass crct10dif_pclmul ghash_clmulni_intel 8021q rapl garp mrp
[ 1413.933098] joydev stp llc mousedev intel_agp psmouse cirrus pcspkr i2c_piix4 intel_gtt mac_hid cfg80211 rfkill dm_multipath dm_mod crypto_user fuse bpf_preload qemu_fw_cfg ip_tables x_tables ext4 crc32c_generic crc16 mbcache jbd2 virtio_net serio_raw virtio_rng net_failover ata_generic atkbd rng_core failover virtio_blk virtio_balloon pata_acpi libps2 vivaldi_fmap crc32_pclmul crc32c_intel aesni_intel virtio_pci crypto_simd virtio_pci_legacy_dev i8042 cryptd virtio_pci_modern_dev ata_piix floppy serio
[ 1413.947630] Unloaded tainted modules: sch_teql():1 netdevsim():1 netdevsim():1 intel_uncore_frequency():1 isst_if_mbox_msr():1 intel_uncore_frequency():1 skx_edac():1 isst_if_mbox_msr():1 intel_uncore_frequency():1 intel_cstate():1 isst_if_mbox_msr():1 intel_uncore_frequency():1 skx_edac():1 intel_uncore():1 intel_cstate():1 intel_uncore_frequency():1 isst_if_mbox_msr():1 skx_edac():1 intel_uncore_frequency():1 intel_uncore():1 skx_edac():1 isst_if_mbox_msr():1 intel_cstate():1 isst_if_mbox_msr():1 intel_uncore_frequency():1 intel_uncore():1 skx_edac():1 intel_cstate():1 skx_edac():1 isst_if_mbox_msr():1 intel_uncore():1 intel_cstate():1 intel_uncore_frequency():1 intel_cstate():1 intel_uncore():1 intel_uncore_frequency():1 skx_edac():1 isst_if_mbox_msr():1 intel_uncore_frequency():1 intel_uncore():1 isst_if_mbox_msr():1 skx_edac():1 intel_cstate():1 intel_uncore_frequency():1 intel_cstate():1 isst_if_mbox_msr():1 intel_uncore():1 skx_edac():1 intel_uncore_frequency():1 intel_cstate():1
[ 1413.955104] isst_if_mbox_msr():1 skx_edac():1 intel_uncore():1 intel_uncore():1 isst_if_mbox_msr():1 skx_edac():1 intel_uncore_frequency():1 intel_cstate():1 skx_edac():1 intel_cstate():1 intel_uncore_frequency():1 isst_if_mbox_msr():1 intel_uncore():1 intel_uncore():1 pcc_cpufreq():1 isst_if_mbox_msr():1 skx_edac():1 intel_uncore_frequency():1 intel_cstate():1 pcc_cpufreq():1 skx_edac():1 intel_uncore():1 intel_uncore_frequency():1 isst_if_mbox_msr():1 intel_cstate():1 acpi_cpufreq():1 isst_if_mbox_msr():1 pcc_cpufreq():1 intel_uncore_frequency():1 intel_cstate():1 intel_uncore():1 skx_edac():1 pcc_cpufreq():1 acpi_cpufreq():1 skx_edac():1 intel_uncore_frequency():1 intel_uncore():1 intel_cstate():1 isst_if_mbox_msr():1 pcc_cpufreq():1 intel_cstate():1 isst_if_mbox_msr():1 pcc_cpufreq():1 skx_edac():1 acpi_cpufreq():1 intel_uncore_frequency():1 intel_uncore():1 acpi_cpufreq():1 isst_if_mbox_msr():1 skx_edac():1 intel_cstate():1 intel_uncore():1 pcc_cpufreq():1
[ 1413.969669] intel_uncore_frequency():1 intel_uncore():1 acpi_cpufreq():1 pcc_cpufreq():1 intel_cstate():1 skx_edac():1 acpi_cpufreq():1 intel_uncore_frequency():1 isst_if_mbox_msr():1 pcc_cpufreq():1 intel_uncore():1 acpi_cpufreq():1 isst_if_mbox_msr():1 pcc_cpufreq():1 intel_cstate():1 intel_uncore_frequency():1 skx_edac():1 acpi_cpufreq():1 skx_edac():1 intel_uncore():1 pcc_cpufreq():1 intel_uncore_frequency():1 intel_cstate():1 intel_uncore_frequency():1 isst_if_mbox_msr():1 pcc_cpufreq():1 acpi_cpufreq():1 intel_uncore_frequency():1 pcc_cpufreq():1 intel_uncore():1 intel_cstate():1 acpi_cpufreq():1 pcc_cpufreq():1 isst_if_mbox_msr():1 intel_uncore_frequency():1 skx_edac():1 isst_if_mbox_msr():1 acpi_cpufreq():1 pcc_cpufreq():1 intel_uncore_frequency():1 intel_uncore():1 isst_if_mbox_msr():1 skx_edac():1 acpi_cpufreq():1 pcc_cpufreq():1 intel_cstate():1 acpi_cpufreq():1 intel_uncore_frequency():1 isst_if_mbox_msr():1 acpi_cpufreq():1 pcc_cpufreq():1 intel_uncore():1
[ 1413.983860] intel_uncore_frequency():1 skx_edac():1 intel_cstate():1 acpi_cpufreq():1 pcc_cpufreq():1 isst_if_mbox_msr():1 intel_uncore_frequency():1 acpi_cpufreq():1 intel_uncore():1 acpi_cpufreq():1 skx_edac():1 isst_if_mbox_msr():1 intel_uncore_frequency():1 intel_cstate():1 pcc_cpufreq():1 acpi_cpufreq():1 intel_uncore_frequency():1 isst_if_mbox_msr():1 intel_cstate():1 intel_uncore():1 pcc_cpufreq():1 skx_edac():1 intel_cstate():1 acpi_cpufreq():1 isst_if_mbox_msr():1 intel_uncore_frequency():1 intel_uncore():1 skx_edac():1 pcc_cpufreq():1 skx_edac():1 pcc_cpufreq():1 intel_uncore_frequency():1 intel_cstate():1 intel_uncore():1 acpi_cpufreq():1 isst_if_mbox_msr():1 intel_uncore_frequency():1 skx_edac():1 isst_if_mbox_msr():1 pcc_cpufreq():1 intel_cstate():1 intel_uncore():1 acpi_cpufreq():1 isst_if_mbox_msr():1 intel_uncore_frequency():1 acpi_cpufreq():1 intel_cstate():1 intel_uncore_frequency():1 pcc_cpufreq():1 intel_uncore():1 skx_edac():1 acpi_cpufreq():1 isst_if_mbox_msr():1
[ 1413.998221] intel_uncore():1 pcc_cpufreq():1 skx_edac():1 acpi_cpufreq():1 intel_uncore_frequency():1 intel_cstate():1 pcc_cpufreq():1 isst_if_mbox_msr():1 skx_edac():1 intel_cstate():1 intel_uncore_frequency():1 acpi_cpufreq():1 intel_uncore():1 skx_edac():1 isst_if_mbox_msr():1 pcc_cpufreq():1 intel_cstate():1 acpi_cpufreq():1 skx_edac():1 isst_if_mbox_msr():1 intel_uncore():1 intel_uncore_frequency():1 skx_edac():1 pcc_cpufreq():1 acpi_cpufreq():1 intel_cstate():1 intel_uncore_frequency():1 intel_uncore():1 isst_if_mbox_msr():1 skx_edac():1 acpi_cpufreq():1 intel_cstate():1 pcc_cpufreq():1 isst_if_mbox_msr():1 intel_uncore_frequency():1 intel_uncore():1 skx_edac():1 intel_uncore_frequency():1 acpi_cpufreq():1 isst_if_mbox_msr():1 intel_uncore():1 intel_cstate():1 pcc_cpufreq():1 skx_edac():1 acpi_cpufreq():1 intel_uncore():1 intel_uncore_frequency():1 skx_edac():1 isst_if_mbox_msr():1 intel_cstate():1 pcc_cpufreq():1 intel_uncore_frequency():1 isst_if_mbox_msr():1 skx_edac():1
[ 1414.012813] intel_cstate():1 intel_uncore():1 intel_uncore_frequency():1 pcc_cpufreq():1 skx_edac():1 isst_if_mbox_msr():1 acpi_cpufreq():1 isst_if_mbox_msr():1 intel_uncore():1 intel_uncore_frequency():1 pcc_cpufreq():1 acpi_cpufreq():1 skx_edac():1 intel_cstate():1 intel_uncore_frequency():1 skx_edac():1 pcc_cpufreq():1 intel_uncore_frequency():1 skx_edac():1 intel_cstate():1 isst_if_mbox_msr():1 acpi_cpufreq():1 intel_uncore():1 intel_cstate():1 pcc_cpufreq():1 intel_uncore_frequency():1 isst_if_mbox_msr():1 intel_cstate():1 pcc_cpufreq():1 acpi_cpufreq():1 intel_uncore():1 skx_edac():1 isst_if_mbox_msr():1 acpi_cpufreq():1 pcc_cpufreq():1 intel_uncore_frequency():1 acpi_cpufreq():1 isst_if_mbox_msr():1 intel_uncore():1 intel_cstate():1 skx_edac():1 intel_uncore():1 skx_edac():1 intel_cstate():1 acpi_cpufreq():1 intel_uncore_frequency():1 isst_if_mbox_msr():1 pcc_cpufreq():1 intel_uncore():1 acpi_cpufreq():1 intel_cstate():1 isst_if_mbox_msr():1 intel_uncore_frequency():1
[ 1414.027301] pcc_cpufreq():1 skx_edac():1 intel_uncore_frequency():1 intel_uncore():1 pcc_cpufreq():1 skx_edac():1 intel_cstate():1 isst_if_mbox_msr():1 acpi_cpufreq():1 isst_if_mbox_msr():1 intel_uncore_frequency():1 intel_uncore():1 skx_edac():1 intel_cstate():1 pcc_cpufreq():1 intel_cstate():1 skx_edac():1 isst_if_mbox_msr():1 pcc_cpufreq():1 skx_edac():1 intel_uncore_frequency():1 intel_cstate():1 acpi_cpufreq():1 intel_uncore():1 acpi_cpufreq():1 isst_if_mbox_msr():1 skx_edac():1 pcc_cpufreq():1 intel_uncore():1 isst_if_mbox_msr():1 intel_uncore():1 intel_uncore_frequency():1 acpi_cpufreq():1 isst_if_mbox_msr():1 intel_uncore_frequency():1 intel_uncore():1 pcc_cpufreq():1 skx_edac():1 intel_cstate():1 isst_if_mbox_msr():1 acpi_cpufreq():1 intel_uncore_frequency():1 skx_edac():1 pcc_cpufreq():1 intel_uncore_frequency():1 isst_if_mbox_msr():1 acpi_cpufreq():1 intel_uncore():1 intel_cstate():1 intel_uncore_frequency():1 skx_edac():1 pcc_cpufreq():1 intel_uncore():1 skx_edac():1
[ 1414.041739] isst_if_mbox_msr():1 acpi_cpufreq():1 pcc_cpufreq():1 intel_cstate():1 isst_if_mbox_msr():1 skx_edac():1 pcc_cpufreq():1 intel_cstate():1 isst_if_mbox_msr():1 acpi_cpufreq():1 pcc_cpufreq():1 skx_edac():1 intel_uncore():1 intel_uncore_frequency():1 intel_cstate():1 intel_uncore_frequency():1 acpi_cpufreq():1 intel_cstate():1 intel_uncore():1 skx_edac():1 pcc_cpufreq():1 isst_if_mbox_msr():1 skx_edac():1 intel_cstate():1 acpi_cpufreq():1 isst_if_mbox_msr():1 intel_uncore():1 pcc_cpufreq():1 intel_uncore_frequency():1 skx_edac():1 intel_uncore_frequency():1 intel_cstate():1 pcc_cpufreq():1 acpi_cpufreq():1 intel_uncore():1 isst_if_mbox_msr():1 intel_uncore():1 acpi_cpufreq():1 isst_if_mbox_msr():1 intel_cstate():1 pcc_cpufreq():1 skx_edac():1 intel_uncore_frequency():1 intel_cstate():1 acpi_cpufreq():1 intel_uncore():1 pcc_cpufreq():1 intel_uncore_frequency():1 isst_if_mbox_msr():1 skx_edac():1 skx_edac():1 pcc_cpufreq():1 intel_cstate():1 intel_uncore_frequency():1
[ 1414.056263] acpi_cpufreq():1 isst_if_mbox_msr():1 skx_edac():1 intel_uncore():1 pcc_cpufreq():1 intel_uncore_frequency():1 acpi_cpufreq():1 intel_cstate():1 skx_edac():1 isst_if_mbox_msr():1 pcc_cpufreq():1 intel_uncore():1 acpi_cpufreq():1 pcc_cpufreq():1 intel_uncore_frequency():1 skx_edac():1 acpi_cpufreq():1 isst_if_mbox_msr():1 intel_uncore():1 pcc_cpufreq():1 intel_cstate():1 intel_uncore_frequency():1 isst_if_mbox_msr():1 intel_uncore():1 acpi_cpufreq():1 intel_cstate():1 pcc_cpufreq():1 skx_edac():1 intel_cstate():1 pcc_cpufreq():1 intel_uncore():1 skx_edac():1 intel_uncore_frequency():1 acpi_cpufreq():1 isst_if_mbox_msr():1 intel_cstate():1 pcc_cpufreq():1 isst_if_mbox_msr():1 intel_uncore():1 skx_edac():1 acpi_cpufreq():1 acpi_cpufreq():1 intel_uncore():1 pcc_cpufreq():1 skx_edac():1 intel_cstate():1 pcc_cpufreq():1 intel_uncore():1 acpi_cpufreq():1 acpi_cpufreq():1 pcc_cpufreq():1 intel_cstate():1 pcc_cpufreq():1 acpi_cpufreq():1 intel_uncore():1 pcc_cpufreq():1
[ 1414.070664] intel_uncore():1 acpi_cpufreq():1 intel_cstate():1 acpi_cpufreq():1 pcc_cpufreq():1 acpi_cpufreq():1 intel_cstate():1 intel_uncore():1 acpi_cpufreq():1 pcc_cpufreq():1 intel_cstate():1 pcc_cpufreq():1 acpi_cpufreq():1 intel_uncore():1 intel_cstate():1 acpi_cpufreq():1 intel_uncore():1 pcc_cpufreq():1 intel_cstate():1 acpi_cpufreq():1 intel_uncore():1 acpi_cpufreq():1 pcc_cpufreq():1 acpi_cpufreq():1 intel_cstate():1 intel_uncore():1 intel_uncore():1 acpi_cpufreq():1 [last unloaded: sch_teql]
[ 1414.092501] CR2: 0000000000000000
[ 1414.093086] ---[ end trace 0000000000000000 ]---
[ 1414.093888] RIP: 0010:0x0
[ 1414.094360] Code: Unable to access opcode bytes at RIP 0xffffffffffffffd6.
[ 1414.095527] RSP: 0018:ffffb29bc1004ee8 EFLAGS: 00010202
[ 1414.096423] RAX: 0000000000000000 RBX: 0000000000000001 RCX: ffff8c57cbc33c78
[ 1414.097630] RDX: 0000000000000000 RSI: ffffb29bc1004ef8 RDI: ffff8c2c45f6edc0
[ 1414.098857] RBP: ffff8c57cbc33b40 R08: 0000000000000000 R09: 0000000000000100
[ 1414.100053] R10: 0000000000000200 R11: ffffffff8aa060c0 R12: ffff8c2dd7ef8000
[ 1414.101256] R13: 0000000000000000 R14: 000000000000000a R15: 0000000000000000
[ 1414.102490] FS: 00007f4819ffb6c0(0000) GS:ffff8c57cbc00000(0000) knlGS:0000000000000000
[ 1414.103848] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 1414.104833] CR2: ffffffffffffffd6 CR3: 00000002a9a94002 CR4: 0000000000770ee0
[ 1414.106056] PKRU: 55555554
[ 1414.106543] Kernel panic - not syncing: Fatal exception in interrupt
[ 1414.108473] Kernel Offset: 0x7e00000 from 0xffffffff81000000 (relocation range: 0xffffffff80000000-0xffffffffbfffffff)
[ 1414.109741] ---[ end Kernel panic - not syncing: Fatal exception in interrupt ]---
```

Not sure if it's related.

Unfortunately, I can't provide a simple and reliable reproducer, as I can reproduce it only with that particular test and not reliably (sometimes it's the first iteration, sometimes it takes an hour or more to reproduce). However, I'd be more than glad to collect more information from one such machine, if possible.


Additional info:
* package version(s)
linux 5.19.7.arch1-1
qemu-common 7.0.0-12

But also spotted with linux 5.19.5.arch1-1 and older.