| Message ID | 20230928231239.2144579-2-twuufnxlz@gmail.com |
|---|---|
| State | Superseded |
| Headers | show |
| Series | riscv: fix out of bounds in walk_stackframe | expand |
On Fri, Sep 29, 2023 at 07:12:40AM +0800, Edward AD wrote: > Add vmalloc and kernel addresses check to prevent invalid access. > > Closes: https://lore.kernel.org/all/20230926105949.1025995-2-twuufnxlz@gmail.com/ > Fixes: 5d8544e2d007 ("RISC-V: Generic library routines and assembly") > Reported-and-test-by: syzbot+8d2757d62d403b2d9275@syzkaller.appspotmail.com > Link: https://lore.kernel.org/all/0000000000000170df0605ccf91a@google.com/T/ > Signed-off-by: Edward AD <twuufnxlz@gmail.com> > --- > arch/riscv/kernel/stacktrace.c | 3 +++ Where are you getting your odd cc: list from? This has nothing to do with serial drivers... greg k-h
On Fri, Sep 29, 2023 at 07:12:40AM +0800, Edward AD wrote: > Add vmalloc and kernel addresses check to prevent invalid access. > > Closes: https://lore.kernel.org/all/20230926105949.1025995-2-twuufnxlz@gmail.com/ > Fixes: 5d8544e2d007 ("RISC-V: Generic library routines and assembly") > Reported-and-test-by: syzbot+8d2757d62d403b2d9275@syzkaller.appspotmail.com > Link: https://lore.kernel.org/all/0000000000000170df0605ccf91a@google.com/T/ > Signed-off-by: Edward AD <twuufnxlz@gmail.com> > --- > arch/riscv/kernel/stacktrace.c | 3 +++ > 1 file changed, 3 insertions(+) > > diff --git a/arch/riscv/kernel/stacktrace.c b/arch/riscv/kernel/stacktrace.c > index 64a9c093aef9..031a4a35c1d0 100644 > --- a/arch/riscv/kernel/stacktrace.c > +++ b/arch/riscv/kernel/stacktrace.c > @@ -54,6 +54,9 @@ void notrace walk_stackframe(struct task_struct *task, struct pt_regs *regs, > break; > /* Unwind stack frame */ > frame = (struct stackframe *)fp - 1; > + if ((is_vmalloc_addr(frame) && !pfn_valid(page_to_pfn(vmalloc_to_page(frame)))) || > + !virt_addr_valid(frame)) > + break; > sp = fp; > if (regs && (regs->epc == pc) && (frame->fp & 0x7)) { > fp = frame->ra; > -- > 2.25.1 > Hi, This is the friendly patch-bot of Greg Kroah-Hartman. You have sent him a patch that has triggered this response. He used to manually respond to these common problems, but in order to save his sanity (he kept writing the same thing over and over, yet to different people), I was created. Hopefully you will not take offence and will fix the problem in your patch and resubmit it so that it can be accepted into the Linux kernel tree. You are receiving this message because of the following common error(s) as indicated below: - Your patch contains warnings and/or errors noticed by the scripts/checkpatch.pl tool. - You have marked a patch with a "Fixes:" tag for a commit that is in an older released kernel, yet you do not have a cc: stable line in the signed-off-by area at all, which means that the patch will not be applied to any older kernel releases. To properly fix this, please follow the documented rules in the Documetnation/process/stable-kernel-rules.rst file for how to resolve this. If you wish to discuss this problem further, or you have questions about how to resolve this issue, please feel free to respond to this email and Greg will reply once he has dug out from the pending patches received from other developers. thanks, greg k-h's patch email bot
Hi Edward, On Fri, Sep 29, 2023 at 1:12 AM Edward AD <twuufnxlz@gmail.com> wrote: > > Add vmalloc and kernel addresses check to prevent invalid access. > > Closes: https://lore.kernel.org/all/20230926105949.1025995-2-twuufnxlz@gmail.com/ > Fixes: 5d8544e2d007 ("RISC-V: Generic library routines and assembly") > Reported-and-test-by: syzbot+8d2757d62d403b2d9275@syzkaller.appspotmail.com > Link: https://lore.kernel.org/all/0000000000000170df0605ccf91a@google.com/T/ > Signed-off-by: Edward AD <twuufnxlz@gmail.com> > --- > arch/riscv/kernel/stacktrace.c | 3 +++ > 1 file changed, 3 insertions(+) > > diff --git a/arch/riscv/kernel/stacktrace.c b/arch/riscv/kernel/stacktrace.c > index 64a9c093aef9..031a4a35c1d0 100644 > --- a/arch/riscv/kernel/stacktrace.c > +++ b/arch/riscv/kernel/stacktrace.c > @@ -54,6 +54,9 @@ void notrace walk_stackframe(struct task_struct *task, struct pt_regs *regs, > break; > /* Unwind stack frame */ > frame = (struct stackframe *)fp - 1; > + if ((is_vmalloc_addr(frame) && !pfn_valid(page_to_pfn(vmalloc_to_page(frame)))) || > + !virt_addr_valid(frame)) > + break; > sp = fp; > if (regs && (regs->epc == pc) && (frame->fp & 0x7)) { > fp = frame->ra; > -- > 2.25.1 > I'm still not convinced this will fix the kasan out-of-bounds accesses, the page can be valid but the read can happen at an offset not initialized and trigger such errors right? I still think there is something weird about the stack frame, as to me this should not happen (but admittedly I don't know much about that).
Hi Edward, On Sat, Sep 30, 2023 at 1:06 AM Edward AD <twuufnxlz@gmail.com> wrote: > > Hi Alexandre, > > On Fri, 29 Sep 2023 10:25:59 +0200 Alexandre Ghiti <alexghiti@rivosinc.com> wrote: > > I'm still not convinced this will fix the kasan out-of-bounds > > accesses, the page can be valid but the read can happen at an offset > > not initialized and trigger such errors right? I still think there is > > something weird about the stack frame, as to me this should not happen > > (but admittedly I don't know much about that). > The added check can confirm that the physical page is invalid (whether it is a > vmalloc allocated page or a slab allocated page), and exit the for loop when it is invalid. Yes, but to me this is not what happens in the bug report you link: | BUG: KASAN: out-of-bounds in walk_stackframe+0x130/0x2f2 arch/riscv/kernel/stacktrace.c:59 | Read of size 8 at addr ff20000006d37c38 by task swapper/1/0 So the read at address ff20000006d37c38 is not "normal" according to KASAN (you can see there is no trap, meaning the physical mapping exists). | The buggy address belongs to the virtual mapping at | [ff20000006d30000, ff20000006d39000) created by: | kernel_clone+0x118/0x896 kernel/fork.c:2909 The virtual address is legitimate since the vma exists ^ | The buggy address belongs to the physical page: | page:ff1c00000250dbc0 refcount:1 mapcount:0 mapping:0000000000000000 index:0x0 pfn:0x9436f And the physical page also exists ^ So I insist, checking that a physical mapping exists to exit the loop is not enough, to me, the error here is that the backtrace goes "too far" at an address where nothing was written before and then KASAN complains about that, again, we don't take any page fault here so it's not a problem of existing physical mapping. > > Perhaps we can trust the test results of syzbot. > > Thanks, > edward
On Sat, Sep 30, 2023 at 10:24 AM Conor Dooley <conor@kernel.org> wrote: > > On Sat, Sep 30, 2023 at 08:13:56AM +0200, Greg KH wrote: > > On Sat, Sep 30, 2023 at 07:05:35AM +0800, Edward AD wrote: > > > On Fri, 29 Sep 2023 08:04:57 +0200 Greg KH wrote: > > > > Where are you getting your odd cc: list from? This has nothing to do > > > > with serial drivers... > > > https://lore.kernel.org/all/0000000000000170df0605ccf91a@google.com/raw > > > > I do not understand this answer. > > AIUI, the original report from syzbot "blamed" the serial maintainers. > Not too sure how it determined that though, given the contents. > blame is too strong a word for that auto-inferred hint :) Yes, the actual problem was in a totally different place, but FWIW here's how it happened: Statistically, stacktrace.c and a number of other generic locations are rather unlikely to actually contain the bug (they can, but the chances are that it's deeper in the call stack), so syzbot, while traversing the stack trace, skipped all frames until [<ffffffff80fa0c7e>] fn_show_state+0x1a/0x22 drivers/tty/vt/keyboard.c:614 [<ffffffff80fa1152>] k_spec drivers/tty/vt/keyboard.c:667 [inline] [<ffffffff80fa1152>] k_spec+0xce/0x102 drivers/tty/vt/keyboard.c:656 [<ffffffff80fa306c>] kbd_keycode drivers/tty/vt/keyboard.c:1524 [inline] [<ffffffff80fa306c>] kbd_event+0x5fa/0xa5e drivers/tty/vt/keyboard.c:1543 Per MAINTAINERS, these locations belonged to "TTY LAYER AND SERIAL DRIVERS". Therefore serial in Cc. The automation actually attributes a lot of reports correctly, but, unfortunately, there are also tricky cases like this.
On Mon, Oct 02, 2023 at 09:13:52AM +0200, Alexandre Ghiti wrote: > Hi Edward, > > On Sat, Sep 30, 2023 at 1:06 AM Edward AD <twuufnxlz@gmail.com> wrote: > > > > Hi Alexandre, > > > > On Fri, 29 Sep 2023 10:25:59 +0200 Alexandre Ghiti <alexghiti@rivosinc.com> wrote: > > > I'm still not convinced this will fix the kasan out-of-bounds > > > accesses, the page can be valid but the read can happen at an offset > > > not initialized and trigger such errors right? I still think there is > > > something weird about the stack frame, as to me this should not happen > > > (but admittedly I don't know much about that). > > The added check can confirm that the physical page is invalid (whether it is a > > vmalloc allocated page or a slab allocated page), and exit the for loop when it is invalid. > > Yes, but to me this is not what happens in the bug report you link: > > | BUG: KASAN: out-of-bounds in walk_stackframe+0x130/0x2f2 > arch/riscv/kernel/stacktrace.c:59 > | Read of size 8 at addr ff20000006d37c38 by task swapper/1/0 > > So the read at address ff20000006d37c38 is not "normal" according to > KASAN (you can see there is no trap, meaning the physical mapping > exists). > > | The buggy address belongs to the virtual mapping at > | [ff20000006d30000, ff20000006d39000) created by: > | kernel_clone+0x118/0x896 kernel/fork.c:2909 > > The virtual address is legitimate since the vma exists ^ > > | The buggy address belongs to the physical page: > | page:ff1c00000250dbc0 refcount:1 mapcount:0 mapping:0000000000000000 > index:0x0 pfn:0x9436f > > And the physical page also exists ^ > > So I insist, checking that a physical mapping exists to exit the loop > is not enough, to me, the error here is that the backtrace goes "too > far" at an address where nothing was written before and then KASAN > complains about that, again, we don't take any page fault here so it's > not a problem of existing physical mapping. Yep! I believe what's happening here is one task unwinding another (starting from whatever gets saved in switch_to()), and there's nothing that prevents that other task from running concurrently and modifying/poisoning its stack. In general trying to unwind a remote stack is racy and broken, but we're stuck with a few bits of the kernel tryingto do that occasionally and so the arch code needs to handle that without blowing up. For KASAN specifically you'll need to access the stack with unchecked accesses (e.g. using READ_ONCE_NOCHECK() to read the struct stackframe), and you'll probably want to add some explicit checks that pointers are within stack bounds since concurrent modification (or corruption) could result in entirely bogus pointers. I *think* that we do the right thing on arm64, so you might want to take a look at arm64's unwinder in arch/arm64/kernel/stacktrace.c, arch/arm64/include/asm/stacktrace.h, and arch/arm64/include/asm/stacktrace/common.h. Mark.
Hi Mark, On 02/10/2023 15:41, Mark Rutland wrote: > On Mon, Oct 02, 2023 at 09:13:52AM +0200, Alexandre Ghiti wrote: >> Hi Edward, >> >> On Sat, Sep 30, 2023 at 1:06 AM Edward AD<twuufnxlz@gmail.com> wrote: >>> Hi Alexandre, >>> >>> On Fri, 29 Sep 2023 10:25:59 +0200 Alexandre Ghiti<alexghiti@rivosinc.com> wrote: >>>> I'm still not convinced this will fix the kasan out-of-bounds >>>> accesses, the page can be valid but the read can happen at an offset >>>> not initialized and trigger such errors right? I still think there is >>>> something weird about the stack frame, as to me this should not happen >>>> (but admittedly I don't know much about that). >>> The added check can confirm that the physical page is invalid (whether it is a >>> vmalloc allocated page or a slab allocated page), and exit the for loop when it is invalid. >> Yes, but to me this is not what happens in the bug report you link: >> >> | BUG: KASAN: out-of-bounds in walk_stackframe+0x130/0x2f2 >> arch/riscv/kernel/stacktrace.c:59 >> | Read of size 8 at addr ff20000006d37c38 by task swapper/1/0 >> >> So the read at address ff20000006d37c38 is not "normal" according to >> KASAN (you can see there is no trap, meaning the physical mapping >> exists). >> >> | The buggy address belongs to the virtual mapping at >> | [ff20000006d30000, ff20000006d39000) created by: >> | kernel_clone+0x118/0x896 kernel/fork.c:2909 >> >> The virtual address is legitimate since the vma exists ^ >> >> | The buggy address belongs to the physical page: >> | page:ff1c00000250dbc0 refcount:1 mapcount:0 mapping:0000000000000000 >> index:0x0 pfn:0x9436f >> >> And the physical page also exists ^ >> >> So I insist, checking that a physical mapping exists to exit the loop >> is not enough, to me, the error here is that the backtrace goes "too >> far" at an address where nothing was written before and then KASAN >> complains about that, again, we don't take any page fault here so it's >> not a problem of existing physical mapping. > Yep! > > I believe what's happening here is one task unwinding another (starting from > whatever gets saved in switch_to()), and there's nothing that prevents that > other task from running concurrently and modifying/poisoning its stack. In > general trying to unwind a remote stack is racy and broken, but we're stuck > with a few bits of the kernel tryingto do that occasionally and so the arch > code needs to handle that without blowing up. Thanks for that, I had already fixed the "imprecise" unwinder (when we don't have a frame pointer) using READ_ONCE_NOCHECK() but I had not this use case in mind, so I'll fix that too. > For KASAN specifically you'll need to access the stack with unchecked accesses > (e.g. using READ_ONCE_NOCHECK() to read the struct stackframe), and you'll > probably want to add some explicit checks that pointers are within stack bounds > since concurrent modification (or corruption) could result in entirely bogus > pointers. > > I *think* that we do the right thing on arm64, so you might want to take a look > at arm64's unwinder in arch/arm64/kernel/stacktrace.c, > arch/arm64/include/asm/stacktrace.h, and > arch/arm64/include/asm/stacktrace/common.h. And I'll check that for the stack bounds check. Thanks again, Alex > > Mark.
diff --git a/arch/riscv/kernel/stacktrace.c b/arch/riscv/kernel/stacktrace.c index 64a9c093aef9..031a4a35c1d0 100644 --- a/arch/riscv/kernel/stacktrace.c +++ b/arch/riscv/kernel/stacktrace.c @@ -54,6 +54,9 @@ void notrace walk_stackframe(struct task_struct *task, struct pt_regs *regs, break; /* Unwind stack frame */ frame = (struct stackframe *)fp - 1; + if ((is_vmalloc_addr(frame) && !pfn_valid(page_to_pfn(vmalloc_to_page(frame)))) || + !virt_addr_valid(frame)) + break; sp = fp; if (regs && (regs->epc == pc) && (frame->fp & 0x7)) { fp = frame->ra;
Add vmalloc and kernel addresses check to prevent invalid access. Closes: https://lore.kernel.org/all/20230926105949.1025995-2-twuufnxlz@gmail.com/ Fixes: 5d8544e2d007 ("RISC-V: Generic library routines and assembly") Reported-and-test-by: syzbot+8d2757d62d403b2d9275@syzkaller.appspotmail.com Link: https://lore.kernel.org/all/0000000000000170df0605ccf91a@google.com/T/ Signed-off-by: Edward AD <twuufnxlz@gmail.com> --- arch/riscv/kernel/stacktrace.c | 3 +++ 1 file changed, 3 insertions(+)