UBUNTU-CVE-2024-27014
In the Linux kernel, the following vulnerability has been resolved: net/mlx5e: Prevent deadlock while disabling aRFS When disabling aRFS under the `priv->state_lock`, any scheduled aRFS works are canceled using the `cancel_work_sync` function, which waits for the work to end if it has already started. However, while waiting for the work handler, the handler will try to acquire the `state_lock` which is already acquired. The worker acquires the lock to delete the rules if the state is down, which is not the worker's responsibility since disabling aRFS deletes the rules. Add an aRFS state variable, which indicates whether the aRFS is enabled and prevent adding rules when the aRFS is disabled. Kernel log: ====================================================== WARNING: possible circular locking dependency detected 6.7.0-rc4_net_next_mlx5_5483eb2 #1 Tainted: G I ------------------------------------------------------ ethtool/386089 is trying to acquire lock: ffff88810f21ce68 ((work_completion)(&rule->arfs_work)){+.+.}-{0:0}, at: __flush_work+0x74/0x4e0 but task is already holding lock: ffff8884a1808cc0 (&priv->state_lock){+.+.}-{3:3}, at: mlx5e_ethtool_set_channels+0x53/0x200 [mlx5_core] which lock already depends on the new lock. the existing dependency chain (in reverse order) is: -> #1 (&priv->state_lock){+.+.}-{3:3}: __mutex_lock+0x80/0xc90 arfs_handle_work+0x4b/0x3b0 [mlx5_core] process_one_work+0x1dc/0x4a0 worker_thread+0x1bf/0x3c0 kthread+0xd7/0x100 ret_from_fork+0x2d/0x50 ret_from_fork_asm+0x11/0x20 -> #0 ((work_completion)(&rule->arfs_work)){+.+.}-{0:0}: __lock_acquire+0x17b4/0x2c80 lock_acquire+0xd0/0x2b0 __flush_work+0x7a/0x4e0 __cancel_work_timer+0x131/0x1c0 arfs_del_rules+0x143/0x1e0 [mlx5_core] mlx5e_arfs_disable+0x1b/0x30 [mlx5_core] mlx5e_ethtool_set_channels+0xcb/0x200 [mlx5_core] ethnl_set_channels+0x28f/0x3b0 ethnl_default_set_doit+0xec/0x240 genl_family_rcv_msg_doit+0xd0/0x120 genl_rcv_msg+0x188/0x2c0 netlink_rcv_skb+0x54/0x100 genl_rcv+0x24/0x40 netlink_unicast+0x1a1/0x270 netlink_sendmsg+0x214/0x460 __sock_sendmsg+0x38/0x60 __sys_sendto+0x113/0x170 __x64_sys_sendto+0x20/0x30 do_syscall_64+0x40/0xe0 entry_SYSCALL_64_after_hwframe+0x46/0x4e other info that might help us debug this: Possible unsafe locking scenario: CPU0 CPU1 ---- ---- lock(&priv->state_lock); lock((work_completion)(&rule->arfs_work)); lock(&priv->state_lock); lock((work_completion)(&rule->arfs_work)); *** DEADLOCK *** 3 locks held by ethtool/386089: #0: ffffffff82ea7210 (cb_lock){++++}-{3:3}, at: genl_rcv+0x15/0x40 #1: ffffffff82e94c88 (rtnl_mutex){+.+.}-{3:3}, at: ethnl_default_set_doit+0xd3/0x240 #2: ffff8884a1808cc0 (&priv->state_lock){+.+.}-{3:3}, at: mlx5e_ethtool_set_channels+0x53/0x200 [mlx5_core] stack backtrace: CPU: 15 PID: 386089 Comm: ethtool Tainted: G I 6.7.0-rc4_net_next_mlx5_5483eb2 #1 Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.13.0-0-gf21b5a4aeb02-prebuilt.qemu.org 04/01/2014 Call Trace: <TASK> dump_stack_lvl+0x60/0xa0 check_noncircular+0x144/0x160 __lock_acquire+0x17b4/0x2c80 lock_acquire+0xd0/0x2b0 ? __flush_work+0x74/0x4e0 ? save_trace+0x3e/0x360 ? __flush_work+0x74/0x4e0 __flush_work+0x7a/0x4e0 ? __flush_work+0x74/0x4e0 ? __lock_acquire+0xa78/0x2c80 ? lock_acquire+0xd0/0x2b0 ? mark_held_locks+0x49/0x70 __cancel_work_timer+0x131/0x1c0 ? mark_held_locks+0x49/0x70 arfs_del_rules+0x143/0x1e0 [mlx5_core] mlx5e_arfs_disable+0x1b/0x30 [mlx5_core] mlx5e_ethtool_set_channels+0xcb/0x200 [mlx5_core] ethnl_set_channels+0x28f/0x3b0 ethnl_default_set_doit+0xec/0x240 genl_family_rcv_msg_doit+0xd0/0x120 genl_rcv_msg+0x188/0x2c0 ? ethn ---truncated---
02 / AFFECTED SOFTWARE
Affected packages
11 explicit affected versions
53 explicit affected versions
116 explicit affected versions
89 explicit affected versions
50 explicit affected versions
6 explicit affected versions
12 explicit affected versions
19 explicit affected versions
6 explicit affected versions
100 explicit affected versions
6 explicit affected versions
85 explicit affected versions
84 explicit affected versions
91 explicit affected versions
86 explicit affected versions
76 explicit affected versions
10 explicit affected versions
5 explicit affected versions
90 explicit affected versions
7 explicit affected versions
84 explicit affected versions
1 explicit affected versions
13 explicit affected versions
85 explicit affected versions
2 explicit affected versions
35 explicit affected versions
51 explicit affected versions
50 explicit affected versions
18 explicit affected versions
42 explicit affected versions
118 explicit affected versions
64 explicit affected versions
6 explicit affected versions
120 explicit affected versions
13 explicit affected versions
4 explicit affected versions
23 explicit affected versions
9 explicit affected versions
124 explicit affected versions
1 explicit affected versions
21 explicit affected versions
16 explicit affected versions
10 explicit affected versions
76 explicit affected versions
7 explicit affected versions
7 explicit affected versions
12 explicit affected versions
111 explicit affected versions
12 explicit affected versions
14 explicit affected versions
1 explicit affected versions
7 explicit affected versions
104 explicit affected versions
12 explicit affected versions
105 explicit affected versions
12 explicit affected versions
45 explicit affected versions
2 explicit affected versions
10 explicit affected versions
37 explicit affected versions
13 explicit affected versions
10 explicit affected versions
68 explicit affected versions
1 explicit affected versions
26 explicit affected versions
13 explicit affected versions
37 explicit affected versions
1 explicit affected versions
29 explicit affected versions
111 explicit affected versions
13 explicit affected versions
96 explicit affected versions
1 explicit affected versions
100 explicit affected versions
92 explicit affected versions
55 explicit affected versions
33 explicit affected versions
23 explicit affected versions
60 explicit affected versions
10 explicit affected versions
11 explicit affected versions
12 explicit affected versions
7 explicit affected versions
12 explicit affected versions
138 explicit affected versions
77 explicit affected versions
132 explicit affected versions
93 explicit affected versions
154 explicit affected versions
10 explicit affected versions
26 explicit affected versions
8 explicit affected versions
1 explicit affected versions
69 explicit affected versions
8 explicit affected versions
13 explicit affected versions
7 explicit affected versions
139 explicit affected versions
78 explicit affected versions
16 explicit affected versions
91 explicit affected versions
8 explicit affected versions
116 explicit affected versions
3 explicit affected versions
132 explicit affected versions
93 explicit affected versions
1 explicit affected versions
2 explicit affected versions
51 explicit affected versions
92 explicit affected versions
5 explicit affected versions
12 explicit affected versions
71 explicit affected versions
80 explicit affected versions
27 explicit affected versions
66 explicit affected versions
121 explicit affected versions
1 explicit affected versions
1 explicit affected versions
98 explicit affected versions
1 explicit affected versions
8 explicit affected versions
55 explicit affected versions
5 explicit affected versions
2 explicit affected versions
8 explicit affected versions
1 explicit affected versions
10 explicit affected versions
4 explicit affected versions
13 explicit affected versions
110 explicit affected versions
13 explicit affected versions
72 explicit affected versions
7 explicit affected versions
140 explicit affected versions
94 explicit affected versions
10 explicit affected versions
12 explicit affected versions
39 explicit affected versions
108 explicit affected versions
80 explicit affected versions
56 explicit affected versions
44 explicit affected versions
16 explicit affected versions
3 explicit affected versions
13 explicit affected versions
26 explicit affected versions
45 explicit affected versions
37 explicit affected versions
8 explicit affected versions
14 explicit affected versions
38 explicit affected versions
167 explicit affected versions
9 explicit affected versions
11 explicit affected versions
92 explicit affected versions
7 explicit affected versions
121 explicit affected versions
4 explicit affected versions
81 explicit affected versions
1 explicit affected versions
1 explicit affected versions
91 explicit affected versions
80 explicit affected versions
79 explicit affected versions
13 explicit affected versions
43 explicit affected versions
77 explicit affected versions
119 explicit affected versions
10 explicit affected versions
8 explicit affected versions
3 explicit affected versions
88 explicit affected versions
14 explicit affected versions
4 explicit affected versions
78 explicit affected versions
11 explicit affected versions
1 explicit affected versions
69 explicit affected versions
123 explicit affected versions
2 explicit affected versions
80 explicit affected versions
03 / CONNECTIONS
Connected vulnerabilities
04 / EVIDENCE
Source records
In the Linux kernel, the following vulnerability has been resolved: net/mlx5e: Prevent deadlock while disabling aRFS When disabling aRFS under the `priv->state_lock`, any scheduled aRFS works are canceled using the `cancel_work_sync` function, which waits for the work to end if it has already started. However, while waiting for the work handler, the handler will try to acquire the `state_lock` which is already acquired. The worker acquires the lock to delete the rules if the state is down, which is not the worker's responsibility since disabling aRFS deletes the rules. Add an aRFS state variable, which indicates whether the aRFS is enabled and prevent adding rules when the aRFS is disabled. Kernel log: ====================================================== WARNING: possible circular locking dependency detected 6.7.0-rc4_net_next_mlx5_5483eb2 #1 Tainted: G I ------------------------------------------------------ ethtool/386089 is trying to acquire lock: ffff88810f21ce68 ((work_completion)(&rule->arfs_work)){+.+.}-{0:0}, at: __flush_work+0x74/0x4e0 but task is already holding lock: ffff8884a1808cc0 (&priv->state_lock){+.+.}-{3:3}, at: mlx5e_ethtool_set_channels+0x53/0x200 [mlx5_core] which lock already depends on the new lock. the existing dependency chain (in reverse order) is: -> #1 (&priv->state_lock){+.+.}-{3:3}: __mutex_lock+0x80/0xc90 arfs_handle_work+0x4b/0x3b0 [mlx5_core] process_one_work+0x1dc/0x4a0 worker_thread+0x1bf/0x3c0 kthread+0xd7/0x100 ret_from_fork+0x2d/0x50 ret_from_fork_asm+0x11/0x20 -> #0 ((work_completion)(&rule->arfs_work)){+.+.}-{0:0}: __lock_acquire+0x17b4/0x2c80 lock_acquire+0xd0/0x2b0 __flush_work+0x7a/0x4e0 __cancel_work_timer+0x131/0x1c0 arfs_del_rules+0x143/0x1e0 [mlx5_core] mlx5e_arfs_disable+0x1b/0x30 [mlx5_core] mlx5e_ethtool_set_channels+0xcb/0x200 [mlx5_core] ethnl_set_channels+0x28f/0x3b0 ethnl_default_set_doit+0xec/0x240 genl_family_rcv_msg_doit+0xd0/0x120 genl_rcv_msg+0x188/0x2c0 netlink_rcv_skb+0x54/0x100 genl_rcv+0x24/0x40 netlink_unicast+0x1a1/0x270 netlink_sendmsg+0x214/0x460 __sock_sendmsg+0x38/0x60 __sys_sendto+0x113/0x170 __x64_sys_sendto+0x20/0x30 do_syscall_64+0x40/0xe0 entry_SYSCALL_64_after_hwframe+0x46/0x4e other info that might help us debug this: Possible unsafe locking scenario: CPU0 CPU1 ---- ---- lock(&priv->state_lock); lock((work_completion)(&rule->arfs_work)); lock(&priv->state_lock); lock((work_completion)(&rule->arfs_work)); *** DEADLOCK *** 3 locks held by ethtool/386089: #0: ffffffff82ea7210 (cb_lock){++++}-{3:3}, at: genl_rcv+0x15/0x40 #1: ffffffff82e94c88 (rtnl_mutex){+.+.}-{3:3}, at: ethnl_default_set_doit+0xd3/0x240 #2: ffff8884a1808cc0 (&priv->state_lock){+.+.}-{3:3}, at: mlx5e_ethtool_set_channels+0x53/0x200 [mlx5_core] stack backtrace: CPU: 15 PID: 386089 Comm: ethtool Tainted: G I 6.7.0-rc4_net_next_mlx5_5483eb2 #1 Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.13.0-0-gf21b5a4aeb02-prebuilt.qemu.org 04/01/2014 Call Trace: <TASK> dump_stack_lvl+0x60/0xa0 check_noncircular+0x144/0x160 __lock_acquire+0x17b4/0x2c80 lock_acquire+0xd0/0x2b0 ? __flush_work+0x74/0x4e0 ? save_trace+0x3e/0x360 ? __flush_work+0x74/0x4e0 __flush_work+0x7a/0x4e0 ? __flush_work+0x74/0x4e0 ? __lock_acquire+0xa78/0x2c80 ? lock_acquire+0xd0/0x2b0 ? mark_held_locks+0x49/0x70 __cancel_work_timer+0x131/0x1c0 ? mark_held_locks+0x49/0x70 arfs_del_rules+0x143/0x1e0 [mlx5_core] mlx5e_arfs_disable+0x1b/0x30 [mlx5_core] mlx5e_ethtool_set_channels+0xcb/0x200 [mlx5_core] ethnl_set_channels+0x28f/0x3b0 ethnl_default_set_doit+0xec/0x240 genl_family_rcv_msg_doit+0xd0/0x120 genl_rcv_msg+0x188/0x2c0 ? ethn ---truncated---
05 / REFERENCES
Further evidence
- https://git.kernel.org/linus/fef965764cf562f28afb997b626fc7c3cec99693
- https://git.kernel.org/stable/c/0080bf99499468030248ebd25dd645e487dcecdc
- https://git.kernel.org/stable/c/46efa4d5930cf3c2af8c01f75e0a47e4fc045e3b
- https://git.kernel.org/stable/c/48c4bb81df19402d4346032353d0795260255e3b
- https://git.kernel.org/stable/c/fef965764cf562f28afb997b626fc7c3cec99693
- https://ubuntu.com/security/CVE-2024-27014
- https://ubuntu.com/security/notices/USN-6893-1
- https://ubuntu.com/security/notices/USN-6893-2
- https://ubuntu.com/security/notices/USN-6893-3
- https://ubuntu.com/security/notices/USN-6918-1
- https://www.cve.org/CVERecord?id=CVE-2024-27014