Steven Rostedt
7a5f01828e
tracing/osnoise: Use a cpumask to know what threads are kthreads
commit 177e1cc2f4 upstream.
The start_kthread() and stop_thread() code was not always called with the
interface_lock held. This means that the kthread variable could be
unexpectedly changed causing the kthread_stop() to be called on it when it
should not have been, leading to:
while true; do
rtla timerlat top -u -q & PID=$!;
sleep 5;
kill -INT $PID;
sleep 0.001;
kill -TERM $PID;
wait $PID;
done
Causing the following OOPS:
Oops: general protection fault, probably for non-canonical address 0xdffffc0000000002: 0000 [#1] PREEMPT SMP KASAN PTI
KASAN: null-ptr-deref in range [0x0000000000000010-0x0000000000000017]
CPU: 5 UID: 0 PID: 885 Comm: timerlatu/5 Not tainted 6.11.0-rc4-test-00002-gbc754cc76d1b-dirty #125 a533010b71dab205ad2f507188ce8c82203b0254
Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS 1.16.3-debian-1.16.3-2 04/01/2014
RIP: 0010:hrtimer_active+0x58/0x300
Code: 48 c1 ee 03 41 54 48 01 d1 48 01 d6 55 53 48 83 ec 20 80 39 00 0f 85 30 02 00 00 49 8b 6f 30 4c 8d 75 10 4c 89 f0 48 c1 e8 03 <0f> b6 3c 10 4c 89 f0 83 e0 07 83 c0 03 40 38 f8 7c 09 40 84 ff 0f
RSP: 0018:ffff88811d97f940 EFLAGS: 00010202
RAX: 0000000000000002 RBX: ffff88823c6b5b28 RCX: ffffed10478d6b6b
RDX: dffffc0000000000 RSI: ffffed10478d6b6c RDI: ffff88823c6b5b28
RBP: 0000000000000000 R08: ffff88823c6b5b58 R09: ffff88823c6b5b60
R10: ffff88811d97f957 R11: 0000000000000010 R12: 00000000000a801d
R13: ffff88810d8b35d8 R14: 0000000000000010 R15: ffff88823c6b5b28
FS: 0000000000000000(0000) GS:ffff88823c680000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 0000561858ad7258 CR3: 000000007729e001 CR4: 0000000000170ef0
Call Trace:
<TASK>
? die_addr+0x40/0xa0
? exc_general_protection+0x154/0x230
? asm_exc_general_protection+0x26/0x30
? hrtimer_active+0x58/0x300
? __pfx_mutex_lock+0x10/0x10
? __pfx_locks_remove_file+0x10/0x10
hrtimer_cancel+0x15/0x40
timerlat_fd_release+0x8e/0x1f0
? security_file_release+0x43/0x80
__fput+0x372/0xb10
task_work_run+0x11e/0x1f0
? _raw_spin_lock+0x85/0xe0
? __pfx_task_work_run+0x10/0x10
? poison_slab_object+0x109/0x170
? do_exit+0x7a0/0x24b0
do_exit+0x7bd/0x24b0
? __pfx_migrate_enable+0x10/0x10
? __pfx_do_exit+0x10/0x10
? __pfx_read_tsc+0x10/0x10
? ktime_get+0x64/0x140
? _raw_spin_lock_irq+0x86/0xe0
do_group_exit+0xb0/0x220
get_signal+0x17ba/0x1b50
? vfs_read+0x179/0xa40
? timerlat_fd_read+0x30b/0x9d0
? __pfx_get_signal+0x10/0x10
? __pfx_timerlat_fd_read+0x10/0x10
arch_do_signal_or_restart+0x8c/0x570
? __pfx_arch_do_signal_or_restart+0x10/0x10
? vfs_read+0x179/0xa40
? ksys_read+0xfe/0x1d0
? __pfx_ksys_read+0x10/0x10
syscall_exit_to_user_mode+0xbc/0x130
do_syscall_64+0x74/0x110
? __pfx___rseq_handle_notify_resume+0x10/0x10
? __pfx_ksys_read+0x10/0x10
? fpregs_restore_userregs+0xdb/0x1e0
? fpregs_restore_userregs+0xdb/0x1e0
? syscall_exit_to_user_mode+0x116/0x130
? do_syscall_64+0x74/0x110
? do_syscall_64+0x74/0x110
? do_syscall_64+0x74/0x110
entry_SYSCALL_64_after_hwframe+0x71/0x79
RIP: 0033:0x7ff0070eca9c
Code: Unable to access opcode bytes at 0x7ff0070eca72.
RSP: 002b:00007ff006dff8c0 EFLAGS: 00000246 ORIG_RAX: 0000000000000000
RAX: 0000000000000000 RBX: 0000000000000005 RCX: 00007ff0070eca9c
RDX: 0000000000000400 RSI: 00007ff006dff9a0 RDI: 0000000000000003
RBP: 00007ff006dffde0 R08: 0000000000000000 R09: 00007ff000000ba0
R10: 00007ff007004b08 R11: 0000000000000246 R12: 0000000000000003
R13: 00007ff006dff9a0 R14: 0000000000000007 R15: 0000000000000008
</TASK>
Modules linked in: snd_hda_intel snd_intel_dspcfg snd_intel_sdw_acpi snd_hda_codec snd_hwdep snd_hda_core
---[ end trace 0000000000000000 ]---
This is because it would mistakenly call kthread_stop() on a user space
thread making it "exit" before it actually exits.
Since kthreads are created based on global behavior, use a cpumask to know
when kthreads are running and that they need to be shutdown before
proceeding to do new work.
Link: https://lore.kernel.org/all/20240820130001.124768-1-tglozar@redhat.com/
This was debugged by using the persistent ring buffer:
Link: https://lore.kernel.org/all/20240823013902.135036960@goodmis.org/
Note, locking was originally used to fix this, but that proved to cause too
many deadlocks to work around:
https://lore.kernel.org/linux-trace-kernel/20240823102816.5e55753b@gandalf.local.home/
Cc: stable@vger.kernel.org
Cc: Masami Hiramatsu <mhiramat@kernel.org>
Cc: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Cc: "Luis Claudio R. Goncalves" <lgoncalv@redhat.com>
Link: https://lore.kernel.org/20240904103428.08efdf4c@gandalf.local.home
Fixes: e88ed227f6 ("tracing/timerlat: Add user-space interface")
Reported-by: Tomas Glozar <tglozar@redhat.com>
Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2024-09-12 11:11:27 +02:00
..
2024-08-29 17:33:58 +02:00
2024-08-29 17:33:24 +02:00
2023-09-05 11:01:47 -07:00
2024-08-03 08:54:34 +02:00
2024-09-08 07:54:32 +02:00
2024-04-03 15:28:50 +02:00
2024-08-03 08:54:36 +02:00
2023-11-20 11:58:53 +01:00
2024-06-27 13:49:13 +02:00
2024-08-19 06:04:24 +02:00
2023-06-09 23:29:50 +10:00
2023-11-20 11:59:25 +01:00
2024-09-12 11:11:25 +02:00
2024-08-14 13:58:53 +02:00
2024-04-17 11:19:26 +02:00
2024-04-13 13:07:29 +02:00
2024-09-08 07:54:44 +02:00
2024-08-29 17:33:24 +02:00
2024-08-29 17:33:41 +02:00
2024-09-12 11:11:27 +02:00
2023-08-30 08:17:35 -07:00
2024-01-31 16:18:49 -08:00
2023-11-28 17:19:56 +00:00
2024-02-05 20:14:14 +00:00
2023-07-21 12:17:25 -04:00
2024-07-11 12:49:18 +02:00
2023-10-13 18:34:46 +02:00
2024-05-02 16:32:50 +02:00
2023-08-08 16:48:47 -04:00
2023-06-05 09:57:20 +02:00
2024-08-29 17:33:30 +02:00
2024-05-02 16:32:47 +02:00
2023-12-20 17:01:51 +01:00
2024-07-11 12:49:15 +02:00
2024-06-21 14:38:47 +02:00
2024-06-21 14:38:40 +02:00
2023-08-21 13:37:28 -07:00
2024-08-14 13:58:38 +02:00
2023-08-29 17:32:32 -07:00
2023-08-25 15:00:36 -07:00
2024-01-05 15:19:41 +01:00
2024-08-14 13:58:57 +02:00
2024-01-31 16:18:56 -08:00
2024-09-12 11:11:27 +02:00
2023-11-28 17:19:40 +00:00
2024-08-14 13:58:51 +02:00
2023-06-08 12:27:20 -07:00
2023-08-24 16:25:14 -07:00
2024-06-12 11:12:52 +02:00
2024-06-12 11:11:50 +02:00
2023-08-21 11:29:12 -07:00
2024-06-12 11:11:50 +02:00
2024-08-14 13:58:59 +02:00
2024-04-13 13:07:29 +02:00
2023-07-10 12:47:01 -07:00
2024-06-21 14:38:50 +02:00
2023-08-21 13:37:59 -07:00
2023-09-19 13:21:33 -07:00
2024-08-14 13:58:47 +02:00
2023-11-28 17:20:04 +00:00
2023-08-18 10:18:55 -07:00
2024-09-12 11:11:25 +02:00
2023-07-14 15:02:57 -07:00
2023-08-17 12:32:15 -07:00
2024-08-03 08:54:13 +02:00
2023-11-28 17:19:36 +00:00
2024-06-12 11:12:52 +02:00
2024-06-12 11:11:27 +02:00
2024-07-05 09:34:04 +02:00
2024-04-03 15:28:54 +02:00
2023-06-28 16:05:21 -07:00
2024-08-03 08:54:16 +02:00
2023-11-28 17:20:08 +00:00
2023-08-15 15:26:17 -07:00
2023-06-29 15:19:43 -07:00
2024-07-11 12:49:10 +02:00
2023-11-28 17:19:40 +00:00
2023-06-19 16:25:28 -07:00
2024-08-03 08:54:29 +02:00
2023-11-28 17:19:57 +00:00
2023-08-07 15:57:22 -10:00
2024-05-17 12:02:31 +02:00