UBUNTU-CVE-2024-26958
In the Linux kernel, the following vulnerability has been resolved: nfs: fix UAF in direct writes In production we have been hitting the following warning consistently ------------[ cut here ]------------ refcount_t: underflow; use-after-free. WARNING: CPU: 17 PID: 1800359 at lib/refcount.c:28 refcount_warn_saturate+0x9c/0xe0 Workqueue: nfsiod nfs_direct_write_schedule_work [nfs] RIP: 0010:refcount_warn_saturate+0x9c/0xe0 PKRU: 55555554 Call Trace: <TASK> ? __warn+0x9f/0x130 ? refcount_warn_saturate+0x9c/0xe0 ? report_bug+0xcc/0x150 ? handle_bug+0x3d/0x70 ? exc_invalid_op+0x16/0x40 ? asm_exc_invalid_op+0x16/0x20 ? refcount_warn_saturate+0x9c/0xe0 nfs_direct_write_schedule_work+0x237/0x250 [nfs] process_one_work+0x12f/0x4a0 worker_thread+0x14e/0x3b0 ? ZSTD_getCParams_internal+0x220/0x220 kthread+0xdc/0x120 ? __btf_name_valid+0xa0/0xa0 ret_from_fork+0x1f/0x30 This is because we're completing the nfs_direct_request twice in a row. The source of this is when we have our commit requests to submit, we process them and send them off, and then in the completion path for the commit requests we have if (nfs_commit_end(cinfo.mds)) nfs_direct_write_complete(dreq); However since we're submitting asynchronous requests we sometimes have one that completes before we submit the next one, so we end up calling complete on the nfs_direct_request twice. The only other place we use nfs_generic_commit_list() is in __nfs_commit_inode, which wraps this call in a nfs_commit_begin(); nfs_commit_end(); Which is a common pattern for this style of completion handling, one that is also repeated in the direct code with get_dreq()/put_dreq() calls around where we process events as well as in the completion paths. Fix this by using the same pattern for the commit requests. Before with my 200 node rocksdb stress running this warning would pop every 10ish minutes. With my patch the stress test has been running for several hours without popping.
02 / AFFECTED SOFTWARE
Affected packages
11 explicit affected versions
53 explicit affected versions
116 explicit affected versions
31 explicit affected versions
45 explicit affected versions
50 explicit affected versions
6 explicit affected versions
12 explicit affected versions
19 explicit affected versions
6 explicit affected versions
54 explicit affected versions
57 explicit affected versions
4 explicit affected versions
40 explicit affected versions
91 explicit affected versions
51 explicit affected versions
40 explicit affected versions
10 explicit affected versions
3 explicit affected versions
47 explicit affected versions
191 explicit affected versions
7 explicit affected versions
46 explicit affected versions
13 explicit affected versions
85 explicit affected versions
35 explicit affected versions
7 explicit affected versions
14 explicit affected versions
18 explicit affected versions
9 explicit affected versions
118 explicit affected versions
34 explicit affected versions
5 explicit affected versions
120 explicit affected versions
13 explicit affected versions
4 explicit affected versions
121 explicit affected versions
23 explicit affected versions
7 explicit affected versions
124 explicit affected versions
1 explicit affected versions
21 explicit affected versions
16 explicit affected versions
10 explicit affected versions
76 explicit affected versions
5 explicit affected versions
7 explicit affected versions
12 explicit affected versions
111 explicit affected versions
38 explicit affected versions
12 explicit affected versions
14 explicit affected versions
7 explicit affected versions
104 explicit affected versions
12 explicit affected versions
105 explicit affected versions
12 explicit affected versions
120 explicit affected versions
10 explicit affected versions
10 explicit affected versions
37 explicit affected versions
13 explicit affected versions
10 explicit affected versions
27 explicit affected versions
1 explicit affected versions
26 explicit affected versions
13 explicit affected versions
144 explicit affected versions
6 explicit affected versions
1 explicit affected versions
29 explicit affected versions
111 explicit affected versions
13 explicit affected versions
51 explicit affected versions
1 explicit affected versions
100 explicit affected versions
49 explicit affected versions
11 explicit affected versions
33 explicit affected versions
23 explicit affected versions
10 explicit affected versions
188 explicit affected versions
11 explicit affected versions
12 explicit affected versions
7 explicit affected versions
12 explicit affected versions
138 explicit affected versions
2 explicit affected versions
39 explicit affected versions
32 explicit affected versions
132 explicit affected versions
48 explicit affected versions
165 explicit affected versions
154 explicit affected versions
10 explicit affected versions
26 explicit affected versions
6 explicit affected versions
1 explicit affected versions
6 explicit affected versions
13 explicit affected versions
5 explicit affected versions
139 explicit affected versions
78 explicit affected versions
16 explicit affected versions
51 explicit affected versions
6 explicit affected versions
40 explicit affected versions
116 explicit affected versions
3 explicit affected versions
132 explicit affected versions
93 explicit affected versions
1 explicit affected versions
2 explicit affected versions
51 explicit affected versions
92 explicit affected versions
3 explicit affected versions
12 explicit affected versions
37 explicit affected versions
27 explicit affected versions
23 explicit affected versions
121 explicit affected versions
1 explicit affected versions
1 explicit affected versions
98 explicit affected versions
8 explicit affected versions
12 explicit affected versions
5 explicit affected versions
8 explicit affected versions
1 explicit affected versions
10 explicit affected versions
2 explicit affected versions
13 explicit affected versions
110 explicit affected versions
13 explicit affected versions
72 explicit affected versions
7 explicit affected versions
140 explicit affected versions
94 explicit affected versions
43 explicit affected versions
10 explicit affected versions
12 explicit affected versions
12 explicit affected versions
108 explicit affected versions
80 explicit affected versions
56 explicit affected versions
44 explicit affected versions
16 explicit affected versions
3 explicit affected versions
13 explicit affected versions
26 explicit affected versions
37 explicit affected versions
37 explicit affected versions
6 explicit affected versions
14 explicit affected versions
38 explicit affected versions
167 explicit affected versions
40 explicit affected versions
9 explicit affected versions
40 explicit affected versions
11 explicit affected versions
48 explicit affected versions
7 explicit affected versions
21 explicit affected versions
97 explicit affected versions
121 explicit affected versions
4 explicit affected versions
1 explicit affected versions
1 explicit affected versions
46 explicit affected versions
13 explicit affected versions
43 explicit affected versions
77 explicit affected versions
119 explicit affected versions
10 explicit affected versions
8 explicit affected versions
1 explicit affected versions
88 explicit affected versions
14 explicit affected versions
4 explicit affected versions
78 explicit affected versions
11 explicit affected versions
1 explicit affected versions
69 explicit affected versions
123 explicit affected versions
03 / CONNECTIONS
Connected vulnerabilities
04 / EVIDENCE
Source records
In the Linux kernel, the following vulnerability has been resolved: nfs: fix UAF in direct writes In production we have been hitting the following warning consistently ------------[ cut here ]------------ refcount_t: underflow; use-after-free. WARNING: CPU: 17 PID: 1800359 at lib/refcount.c:28 refcount_warn_saturate+0x9c/0xe0 Workqueue: nfsiod nfs_direct_write_schedule_work [nfs] RIP: 0010:refcount_warn_saturate+0x9c/0xe0 PKRU: 55555554 Call Trace: <TASK> ? __warn+0x9f/0x130 ? refcount_warn_saturate+0x9c/0xe0 ? report_bug+0xcc/0x150 ? handle_bug+0x3d/0x70 ? exc_invalid_op+0x16/0x40 ? asm_exc_invalid_op+0x16/0x20 ? refcount_warn_saturate+0x9c/0xe0 nfs_direct_write_schedule_work+0x237/0x250 [nfs] process_one_work+0x12f/0x4a0 worker_thread+0x14e/0x3b0 ? ZSTD_getCParams_internal+0x220/0x220 kthread+0xdc/0x120 ? __btf_name_valid+0xa0/0xa0 ret_from_fork+0x1f/0x30 This is because we're completing the nfs_direct_request twice in a row. The source of this is when we have our commit requests to submit, we process them and send them off, and then in the completion path for the commit requests we have if (nfs_commit_end(cinfo.mds)) nfs_direct_write_complete(dreq); However since we're submitting asynchronous requests we sometimes have one that completes before we submit the next one, so we end up calling complete on the nfs_direct_request twice. The only other place we use nfs_generic_commit_list() is in __nfs_commit_inode, which wraps this call in a nfs_commit_begin(); nfs_commit_end(); Which is a common pattern for this style of completion handling, one that is also repeated in the direct code with get_dreq()/put_dreq() calls around where we process events as well as in the completion paths. Fix this by using the same pattern for the commit requests. Before with my 200 node rocksdb stress running this warning would pop every 10ish minutes. With my patch the stress test has been running for several hours without popping.
05 / REFERENCES
Further evidence
- https://git.kernel.org/linus/17f46b803d4f23c66cacce81db35fef3adb8f2af
- https://git.kernel.org/stable/c/17f46b803d4f23c66cacce81db35fef3adb8f2af
- https://git.kernel.org/stable/c/1daf52b5ffb24870fbeda20b4967526d8f9e12ab
- https://git.kernel.org/stable/c/3abc2d160ed8213948b147295d77d44a22c88fa3
- https://git.kernel.org/stable/c/4595d90b5d2ea5fa4d318d13f59055aa4bf3e7f5
- https://git.kernel.org/stable/c/80d24b308b7ee7037fc90d8ac99f6f78df0a256f
- https://git.kernel.org/stable/c/cf54f66e1dd78990ec6b32177bca7e6ea2144a95
- https://git.kernel.org/stable/c/e25447c35f8745337ea8bc0c9697fcac14df8605
- https://ubuntu.com/security/CVE-2024-26958
- https://ubuntu.com/security/notices/USN-6816-1
- https://ubuntu.com/security/notices/USN-6817-1
- https://ubuntu.com/security/notices/USN-6817-2
- https://ubuntu.com/security/notices/USN-6817-3
- https://ubuntu.com/security/notices/USN-6878-1
- https://ubuntu.com/security/notices/USN-6898-1
- https://ubuntu.com/security/notices/USN-6898-2
- https://ubuntu.com/security/notices/USN-6898-3
- https://ubuntu.com/security/notices/USN-6898-4
- https://ubuntu.com/security/notices/USN-6917-1
- https://ubuntu.com/security/notices/USN-6919-1
- https://ubuntu.com/security/notices/USN-6927-1
- https://ubuntu.com/security/notices/USN-7019-1
- https://www.cve.org/CVERecord?id=CVE-2024-26958