UBUNTU-CVE-2021-46939
In the Linux kernel, the following vulnerability has been resolved: tracing: Restructure trace_clock_global() to never block It was reported that a fix to the ring buffer recursion detection would cause a hung machine when performing suspend / resume testing. The following backtrace was extracted from debugging that case: Call Trace: trace_clock_global+0x91/0xa0 __rb_reserve_next+0x237/0x460 ring_buffer_lock_reserve+0x12a/0x3f0 trace_buffer_lock_reserve+0x10/0x50 __trace_graph_return+0x1f/0x80 trace_graph_return+0xb7/0xf0 ? trace_clock_global+0x91/0xa0 ftrace_return_to_handler+0x8b/0xf0 ? pv_hash+0xa0/0xa0 return_to_handler+0x15/0x30 ? ftrace_graph_caller+0xa0/0xa0 ? trace_clock_global+0x91/0xa0 ? __rb_reserve_next+0x237/0x460 ? ring_buffer_lock_reserve+0x12a/0x3f0 ? trace_event_buffer_lock_reserve+0x3c/0x120 ? trace_event_buffer_reserve+0x6b/0xc0 ? trace_event_raw_event_device_pm_callback_start+0x125/0x2d0 ? dpm_run_callback+0x3b/0xc0 ? pm_ops_is_empty+0x50/0x50 ? platform_get_irq_byname_optional+0x90/0x90 ? trace_device_pm_callback_start+0x82/0xd0 ? dpm_run_callback+0x49/0xc0 With the following RIP: RIP: 0010:native_queued_spin_lock_slowpath+0x69/0x200 Since the fix to the recursion detection would allow a single recursion to happen while tracing, this lead to the trace_clock_global() taking a spin lock and then trying to take it again: ring_buffer_lock_reserve() { trace_clock_global() { arch_spin_lock() { queued_spin_lock_slowpath() { /* lock taken */ (something else gets traced by function graph tracer) ring_buffer_lock_reserve() { trace_clock_global() { arch_spin_lock() { queued_spin_lock_slowpath() { /* DEAD LOCK! */ Tracing should *never* block, as it can lead to strange lockups like the above. Restructure the trace_clock_global() code to instead of simply taking a lock to update the recorded "prev_time" simply use it, as two events happening on two different CPUs that calls this at the same time, really doesn't matter which one goes first. Use a trylock to grab the lock for updating the prev_time, and if it fails, simply try again the next time. If it failed to be taken, that means something else is already updating it. Bugzilla: https://bugzilla.kernel.org/show_bug.cgi?id=212761
02 / AFFECTED SOFTWARE
Affected packages
50 explicit affected versions
6 explicit affected versions
12 explicit affected versions
6 explicit affected versions
1 explicit affected versions
191 explicit affected versions
7 explicit affected versions
67 explicit affected versions
27 explicit affected versions
47 explicit affected versions
20 explicit affected versions
35 explicit affected versions
18 explicit affected versions
77 explicit affected versions
13 explicit affected versions
4 explicit affected versions
23 explicit affected versions
96 explicit affected versions
1 explicit affected versions
1 explicit affected versions
21 explicit affected versions
16 explicit affected versions
10 explicit affected versions
1 explicit affected versions
7 explicit affected versions
12 explicit affected versions
12 explicit affected versions
1 explicit affected versions
7 explicit affected versions
12 explicit affected versions
29 explicit affected versions
37 explicit affected versions
12 explicit affected versions
29 explicit affected versions
22 explicit affected versions
10 explicit affected versions
162 explicit affected versions
37 explicit affected versions
95 explicit affected versions
10 explicit affected versions
1 explicit affected versions
26 explicit affected versions
13 explicit affected versions
1 explicit affected versions
29 explicit affected versions
13 explicit affected versions
61 explicit affected versions
1 explicit affected versions
32 explicit affected versions
33 explicit affected versions
23 explicit affected versions
10 explicit affected versions
46 explicit affected versions
11 explicit affected versions
12 explicit affected versions
69 explicit affected versions
7 explicit affected versions
12 explicit affected versions
29 explicit affected versions
2 explicit affected versions
25 explicit affected versions
10 explicit affected versions
26 explicit affected versions
13 explicit affected versions
16 explicit affected versions
3 explicit affected versions
1 explicit affected versions
51 explicit affected versions
23 explicit affected versions
12 explicit affected versions
27 explicit affected versions
1 explicit affected versions
1 explicit affected versions
26 explicit affected versions
8 explicit affected versions
5 explicit affected versions
19 explicit affected versions
8 explicit affected versions
1 explicit affected versions
20 explicit affected versions
10 explicit affected versions
13 explicit affected versions
1 explicit affected versions
7 explicit affected versions
23 explicit affected versions
10 explicit affected versions
12 explicit affected versions
18 explicit affected versions
44 explicit affected versions
16 explicit affected versions
3 explicit affected versions
13 explicit affected versions
26 explicit affected versions
37 explicit affected versions
38 explicit affected versions
9 explicit affected versions
11 explicit affected versions
139 explicit affected versions
19 explicit affected versions
7 explicit affected versions
72 explicit affected versions
3 explicit affected versions
77 explicit affected versions
54 explicit affected versions
4 explicit affected versions
45 explicit affected versions
19 explicit affected versions
43 explicit affected versions
1 explicit affected versions
119 explicit affected versions
92 explicit affected versions
10 explicit affected versions
8 explicit affected versions
14 explicit affected versions
22 explicit affected versions
4 explicit affected versions
9 explicit affected versions
1 explicit affected versions
1 explicit affected versions
03 / CONNECTIONS
Connected vulnerabilities
04 / EVIDENCE
Source records
In the Linux kernel, the following vulnerability has been resolved: tracing: Restructure trace_clock_global() to never block It was reported that a fix to the ring buffer recursion detection would cause a hung machine when performing suspend / resume testing. The following backtrace was extracted from debugging that case: Call Trace: trace_clock_global+0x91/0xa0 __rb_reserve_next+0x237/0x460 ring_buffer_lock_reserve+0x12a/0x3f0 trace_buffer_lock_reserve+0x10/0x50 __trace_graph_return+0x1f/0x80 trace_graph_return+0xb7/0xf0 ? trace_clock_global+0x91/0xa0 ftrace_return_to_handler+0x8b/0xf0 ? pv_hash+0xa0/0xa0 return_to_handler+0x15/0x30 ? ftrace_graph_caller+0xa0/0xa0 ? trace_clock_global+0x91/0xa0 ? __rb_reserve_next+0x237/0x460 ? ring_buffer_lock_reserve+0x12a/0x3f0 ? trace_event_buffer_lock_reserve+0x3c/0x120 ? trace_event_buffer_reserve+0x6b/0xc0 ? trace_event_raw_event_device_pm_callback_start+0x125/0x2d0 ? dpm_run_callback+0x3b/0xc0 ? pm_ops_is_empty+0x50/0x50 ? platform_get_irq_byname_optional+0x90/0x90 ? trace_device_pm_callback_start+0x82/0xd0 ? dpm_run_callback+0x49/0xc0 With the following RIP: RIP: 0010:native_queued_spin_lock_slowpath+0x69/0x200 Since the fix to the recursion detection would allow a single recursion to happen while tracing, this lead to the trace_clock_global() taking a spin lock and then trying to take it again: ring_buffer_lock_reserve() { trace_clock_global() { arch_spin_lock() { queued_spin_lock_slowpath() { /* lock taken */ (something else gets traced by function graph tracer) ring_buffer_lock_reserve() { trace_clock_global() { arch_spin_lock() { queued_spin_lock_slowpath() { /* DEAD LOCK! */ Tracing should *never* block, as it can lead to strange lockups like the above. Restructure the trace_clock_global() code to instead of simply taking a lock to update the recorded "prev_time" simply use it, as two events happening on two different CPUs that calls this at the same time, really doesn't matter which one goes first. Use a trylock to grab the lock for updating the prev_time, and if it fails, simply try again the next time. If it failed to be taken, that means something else is already updating it. Bugzilla: https://bugzilla.kernel.org/show_bug.cgi?id=212761
05 / REFERENCES
Further evidence
- https://git.kernel.org/linus/aafe104aa9096827a429bc1358f8260ee565b7cc
- https://git.kernel.org/stable/c/1fca00920327be96f3318224f502e4d5460f9545
- https://git.kernel.org/stable/c/2a1bd74b8186d7938bf004f5603f25b84785f63e
- https://git.kernel.org/stable/c/6e2418576228eeb12e7ba82edb8f9500623942ff
- https://git.kernel.org/stable/c/859b47a43f5a0e5b9a92b621dc6ceaad39fb5c8b
- https://git.kernel.org/stable/c/91ca6f6a91f679c8645d7f3307e03ce86ad518c4
- https://git.kernel.org/stable/c/a33614d52e97fc8077eb0b292189ca7d964cc534
- https://git.kernel.org/stable/c/aafe104aa9096827a429bc1358f8260ee565b7cc
- https://git.kernel.org/stable/c/c64da3294a7d59a4bf6874c664c13be892f15f44
- https://git.kernel.org/stable/c/d43d56dbf452ccecc1ec735cd4b6840118005d7c
- https://ubuntu.com/security/CVE-2021-46939
- https://ubuntu.com/security/notices/USN-6778-1
- https://www.cve.org/CVERecord?id=CVE-2021-46939