UBUNTU-CVE-2024-26868
In the Linux kernel, the following vulnerability has been resolved: nfs: fix panic when nfs4_ff_layout_prepare_ds() fails We've been seeing the following panic in production BUG: kernel NULL pointer dereference, address: 0000000000000065 PGD 2f485f067 P4D 2f485f067 PUD 2cc5d8067 PMD 0 RIP: 0010:ff_layout_cancel_io+0x3a/0x90 [nfs_layout_flexfiles] Call Trace: <TASK> ? __die+0x78/0xc0 ? page_fault_oops+0x286/0x380 ? __rpc_execute+0x2c3/0x470 [sunrpc] ? rpc_new_task+0x42/0x1c0 [sunrpc] ? exc_page_fault+0x5d/0x110 ? asm_exc_page_fault+0x22/0x30 ? ff_layout_free_layoutreturn+0x110/0x110 [nfs_layout_flexfiles] ? ff_layout_cancel_io+0x3a/0x90 [nfs_layout_flexfiles] ? ff_layout_cancel_io+0x6f/0x90 [nfs_layout_flexfiles] pnfs_mark_matching_lsegs_return+0x1b0/0x360 [nfsv4] pnfs_error_mark_layout_for_return+0x9e/0x110 [nfsv4] ? ff_layout_send_layouterror+0x50/0x160 [nfs_layout_flexfiles] nfs4_ff_layout_prepare_ds+0x11f/0x290 [nfs_layout_flexfiles] ff_layout_pg_init_write+0xf0/0x1f0 [nfs_layout_flexfiles] __nfs_pageio_add_request+0x154/0x6c0 [nfs] nfs_pageio_add_request+0x26b/0x380 [nfs] nfs_do_writepage+0x111/0x1e0 [nfs] nfs_writepages_callback+0xf/0x30 [nfs] write_cache_pages+0x17f/0x380 ? nfs_pageio_init_write+0x50/0x50 [nfs] ? nfs_writepages+0x6d/0x210 [nfs] ? nfs_writepages+0x6d/0x210 [nfs] nfs_writepages+0x125/0x210 [nfs] do_writepages+0x67/0x220 ? generic_perform_write+0x14b/0x210 filemap_fdatawrite_wbc+0x5b/0x80 file_write_and_wait_range+0x6d/0xc0 nfs_file_fsync+0x81/0x170 [nfs] ? nfs_file_mmap+0x60/0x60 [nfs] __x64_sys_fsync+0x53/0x90 do_syscall_64+0x3d/0x90 entry_SYSCALL_64_after_hwframe+0x46/0xb0 Inspecting the core with drgn I was able to pull this >>> prog.crashed_thread().stack_trace()[0] #0 at 0xffffffffa079657a (ff_layout_cancel_io+0x3a/0x84) in ff_layout_cancel_io at fs/nfs/flexfilelayout/flexfilelayout.c:2021:27 >>> prog.crashed_thread().stack_trace()[0]['idx'] (u32)1 >>> prog.crashed_thread().stack_trace()[0]['flseg'].mirror_array[1].mirror_ds (struct nfs4_ff_layout_ds *)0xffffffffffffffed This is clear from the stack trace, we call nfs4_ff_layout_prepare_ds() which could error out initializing the mirror_ds, and then we go to clean it all up and our check is only for if (!mirror->mirror_ds). This is inconsistent with the rest of the users of mirror_ds, which have if (IS_ERR_OR_NULL(mirror_ds)) to keep from tripping over this exact scenario. Fix this up in ff_layout_cancel_io() to make sure we don't panic when we get an error. I also spot checked all the other instances of checking mirror_ds and we appear to be doing the correct checks everywhere, only unconditionally dereferencing mirror_ds when we know it would be valid.
02 / AFFECTED SOFTWARE
Affected packages
11 explicit affected versions
50 explicit affected versions
6 explicit affected versions
12 explicit affected versions
19 explicit affected versions
6 explicit affected versions
4 explicit affected versions
10 explicit affected versions
3 explicit affected versions
7 explicit affected versions
13 explicit affected versions
35 explicit affected versions
18 explicit affected versions
5 explicit affected versions
13 explicit affected versions
4 explicit affected versions
23 explicit affected versions
7 explicit affected versions
21 explicit affected versions
16 explicit affected versions
10 explicit affected versions
5 explicit affected versions
7 explicit affected versions
12 explicit affected versions
12 explicit affected versions
14 explicit affected versions
1 explicit affected versions
7 explicit affected versions
12 explicit affected versions
12 explicit affected versions
37 explicit affected versions
13 explicit affected versions
10 explicit affected versions
26 explicit affected versions
13 explicit affected versions
29 explicit affected versions
13 explicit affected versions
33 explicit affected versions
23 explicit affected versions
10 explicit affected versions
12 explicit affected versions
7 explicit affected versions
12 explicit affected versions
10 explicit affected versions
26 explicit affected versions
6 explicit affected versions
6 explicit affected versions
13 explicit affected versions
5 explicit affected versions
16 explicit affected versions
6 explicit affected versions
3 explicit affected versions
51 explicit affected versions
3 explicit affected versions
12 explicit affected versions
27 explicit affected versions
8 explicit affected versions
5 explicit affected versions
8 explicit affected versions
10 explicit affected versions
2 explicit affected versions
13 explicit affected versions
13 explicit affected versions
7 explicit affected versions
10 explicit affected versions
12 explicit affected versions
44 explicit affected versions
16 explicit affected versions
3 explicit affected versions
13 explicit affected versions
26 explicit affected versions
37 explicit affected versions
6 explicit affected versions
14 explicit affected versions
38 explicit affected versions
9 explicit affected versions
11 explicit affected versions
7 explicit affected versions
4 explicit affected versions
13 explicit affected versions
43 explicit affected versions
10 explicit affected versions
8 explicit affected versions
1 explicit affected versions
14 explicit affected versions
4 explicit affected versions
11 explicit affected versions
1 explicit affected versions
03 / CONNECTIONS
Connected vulnerabilities
04 / EVIDENCE
Source records
In the Linux kernel, the following vulnerability has been resolved: nfs: fix panic when nfs4_ff_layout_prepare_ds() fails We've been seeing the following panic in production BUG: kernel NULL pointer dereference, address: 0000000000000065 PGD 2f485f067 P4D 2f485f067 PUD 2cc5d8067 PMD 0 RIP: 0010:ff_layout_cancel_io+0x3a/0x90 [nfs_layout_flexfiles] Call Trace: <TASK> ? __die+0x78/0xc0 ? page_fault_oops+0x286/0x380 ? __rpc_execute+0x2c3/0x470 [sunrpc] ? rpc_new_task+0x42/0x1c0 [sunrpc] ? exc_page_fault+0x5d/0x110 ? asm_exc_page_fault+0x22/0x30 ? ff_layout_free_layoutreturn+0x110/0x110 [nfs_layout_flexfiles] ? ff_layout_cancel_io+0x3a/0x90 [nfs_layout_flexfiles] ? ff_layout_cancel_io+0x6f/0x90 [nfs_layout_flexfiles] pnfs_mark_matching_lsegs_return+0x1b0/0x360 [nfsv4] pnfs_error_mark_layout_for_return+0x9e/0x110 [nfsv4] ? ff_layout_send_layouterror+0x50/0x160 [nfs_layout_flexfiles] nfs4_ff_layout_prepare_ds+0x11f/0x290 [nfs_layout_flexfiles] ff_layout_pg_init_write+0xf0/0x1f0 [nfs_layout_flexfiles] __nfs_pageio_add_request+0x154/0x6c0 [nfs] nfs_pageio_add_request+0x26b/0x380 [nfs] nfs_do_writepage+0x111/0x1e0 [nfs] nfs_writepages_callback+0xf/0x30 [nfs] write_cache_pages+0x17f/0x380 ? nfs_pageio_init_write+0x50/0x50 [nfs] ? nfs_writepages+0x6d/0x210 [nfs] ? nfs_writepages+0x6d/0x210 [nfs] nfs_writepages+0x125/0x210 [nfs] do_writepages+0x67/0x220 ? generic_perform_write+0x14b/0x210 filemap_fdatawrite_wbc+0x5b/0x80 file_write_and_wait_range+0x6d/0xc0 nfs_file_fsync+0x81/0x170 [nfs] ? nfs_file_mmap+0x60/0x60 [nfs] __x64_sys_fsync+0x53/0x90 do_syscall_64+0x3d/0x90 entry_SYSCALL_64_after_hwframe+0x46/0xb0 Inspecting the core with drgn I was able to pull this >>> prog.crashed_thread().stack_trace()[0] #0 at 0xffffffffa079657a (ff_layout_cancel_io+0x3a/0x84) in ff_layout_cancel_io at fs/nfs/flexfilelayout/flexfilelayout.c:2021:27 >>> prog.crashed_thread().stack_trace()[0]['idx'] (u32)1 >>> prog.crashed_thread().stack_trace()[0]['flseg'].mirror_array[1].mirror_ds (struct nfs4_ff_layout_ds *)0xffffffffffffffed This is clear from the stack trace, we call nfs4_ff_layout_prepare_ds() which could error out initializing the mirror_ds, and then we go to clean it all up and our check is only for if (!mirror->mirror_ds). This is inconsistent with the rest of the users of mirror_ds, which have if (IS_ERR_OR_NULL(mirror_ds)) to keep from tripping over this exact scenario. Fix this up in ff_layout_cancel_io() to make sure we don't panic when we get an error. I also spot checked all the other instances of checking mirror_ds and we appear to be doing the correct checks everywhere, only unconditionally dereferencing mirror_ds when we know it would be valid.
05 / REFERENCES
Further evidence
- https://git.kernel.org/linus/719fcafe07c12646691bd62d7f8d94d657fa0766
- https://git.kernel.org/stable/c/31db25e3141b20e2a76a9f219eeca52e3cab126c
- https://git.kernel.org/stable/c/5ada9016b1217498fad876a3d5b07645cc955608
- https://git.kernel.org/stable/c/719fcafe07c12646691bd62d7f8d94d657fa0766
- https://git.kernel.org/stable/c/7ca651b4ec4a049f5a46a0e5ff921b86b91c47c5
- https://git.kernel.org/stable/c/dac068f164ad05b35e7c0be13f138c3f6adca58f
- https://ubuntu.com/security/CVE-2024-26868
- https://ubuntu.com/security/notices/USN-6816-1
- https://ubuntu.com/security/notices/USN-6817-1
- https://ubuntu.com/security/notices/USN-6817-2
- https://ubuntu.com/security/notices/USN-6817-3
- https://ubuntu.com/security/notices/USN-6878-1
- https://www.cve.org/CVERecord?id=CVE-2024-26868