diff mbox series

[v4,18/36] arm64/gcs: Context switch GCS state for EL0

Message ID 20230807-arm64-gcs-v4-18-68cfa37f9069@kernel.org
State Superseded
Headers show
Series arm64/gcs: Provide support for GCS in userspace | expand

Commit Message

Mark Brown Aug. 7, 2023, 10 p.m. UTC
There are two registers controlling the GCS state of EL0, GCSPR_EL0 which
is the current GCS pointer and GCSCRE0_EL1 which has enable bits for the
specific GCS functionality enabled for EL0. Manage these on context switch
and process lifetime events, GCS is reset on exec().  Also ensure that
any changes to the GCS memory are visible to other PEs and that changes
from other PEs are visible on this one by issuing a GCSB DSYNC when
moving to or from a thread with GCS.

Since the current GCS configuration of a thread will be visible to
userspace we store the configuration in the format used with userspace
and provide a helper which configures the system register as needed.

On systems that support GCS we always allow access to GCSPR_EL0, this
facilitates reporting of GCS faults if userspace implements disabling of
GCS on error - the GCS can still be discovered and examined even if GCS
has been disabled.

Signed-off-by: Mark Brown <broonie@kernel.org>
---
 arch/arm64/include/asm/gcs.h       | 24 +++++++++++++++++
 arch/arm64/include/asm/processor.h |  6 +++++
 arch/arm64/kernel/process.c        | 55 ++++++++++++++++++++++++++++++++++++++
 arch/arm64/mm/Makefile             |  1 +
 arch/arm64/mm/gcs.c                | 39 +++++++++++++++++++++++++++
 5 files changed, 125 insertions(+)

Comments

Catalin Marinas Aug. 11, 2023, 3:32 p.m. UTC | #1
On Mon, Aug 07, 2023 at 11:00:23PM +0100, Mark Brown wrote:
> @@ -271,12 +272,31 @@ static void flush_tagged_addr_state(void)
>  		clear_thread_flag(TIF_TAGGED_ADDR);
>  }
>  
> +#ifdef CONFIG_ARM64_GCS
> +
> +static void flush_gcs(void)
> +{
> +	if (system_supports_gcs()) {

Nitpick: use "if (system_supports_gcs()) return" when we have more than
a line in the conditional block (slightly more consistent with other
places).

> +		gcs_free(current);
> +		current->thread.gcs_el0_mode = 0;
> +		write_sysreg_s(0, SYS_GCSCRE0_EL1);
> +		write_sysreg_s(0, SYS_GCSPR_EL0);
> +	}
> +}

Do we need and isb() or there's one on this path? If it's only EL0
making use of this register, we should be fine with the ERET before
returning to user. Not sure whether the kernel uses this, GCSSTTR
doesn't need it.

> +static void gcs_thread_switch(struct task_struct *next)
> +{
> +	if (!system_supports_gcs())
> +		return;
> +
> +	gcs_preserve_current_state();
> +
> +	/*
> +	 * Ensure that GCS changes are observable by/from other PEs in
> +	 * case of migration.
> +	 */
> +	if (task_gcs_el0_enabled(current) || task_gcs_el0_enabled(next))
> +		gcsb_dsync();

What's this barrier for? The spec (at least the version I have) only
talks about accesses, nothing to do with the registers that we context
switch here.
Mark Brown Aug. 16, 2023, 6:15 p.m. UTC | #2
On Fri, Aug 11, 2023 at 04:32:10PM +0100, Catalin Marinas wrote:
> On Mon, Aug 07, 2023 at 11:00:23PM +0100, Mark Brown wrote:

> > +		gcs_free(current);
> > +		current->thread.gcs_el0_mode = 0;
> > +		write_sysreg_s(0, SYS_GCSCRE0_EL1);
> > +		write_sysreg_s(0, SYS_GCSPR_EL0);
> > +	}
> > +}

> Do we need and isb() or there's one on this path? If it's only EL0
> making use of this register, we should be fine with the ERET before
> returning to user. Not sure whether the kernel uses this, GCSSTTR
> doesn't need it.

They're only used by EL0, at EL1 we do read GCSPR for signal handling
but AIUI that shouldn't be any more of an issue than it is for the
TPIDRs which we don't have a barrier for.  It's possible I'm
misunderstanding though.

> > +	/*
> > +	 * Ensure that GCS changes are observable by/from other PEs in
> > +	 * case of migration.
> > +	 */
> > +	if (task_gcs_el0_enabled(current) || task_gcs_el0_enabled(next))
> > +		gcsb_dsync();

> What's this barrier for? The spec (at least the version I have) only
> talks about accesses, nothing to do with the registers that we context
> switch here.

Right, it's for the GCS memory rather than the registers.  I'm fairly
sure it's excessive but but was erring on the side of caution until I
have convinced myself that the interactions between GCS barriers and
regular barriers were doing the right thing, until we have physical
implementations to contend with I'd guess the practical impact will be
minimal.
Catalin Marinas Aug. 22, 2023, 4:34 p.m. UTC | #3
On Wed, Aug 16, 2023 at 07:15:53PM +0100, Mark Brown wrote:
> On Fri, Aug 11, 2023 at 04:32:10PM +0100, Catalin Marinas wrote:
> > On Mon, Aug 07, 2023 at 11:00:23PM +0100, Mark Brown wrote:
> 
> > > +		gcs_free(current);
> > > +		current->thread.gcs_el0_mode = 0;
> > > +		write_sysreg_s(0, SYS_GCSCRE0_EL1);
> > > +		write_sysreg_s(0, SYS_GCSPR_EL0);
> > > +	}
> > > +}
> 
> > Do we need and isb() or there's one on this path? If it's only EL0
> > making use of this register, we should be fine with the ERET before
> > returning to user. Not sure whether the kernel uses this, GCSSTTR
> > doesn't need it.
> 
> They're only used by EL0, at EL1 we do read GCSPR for signal handling
> but AIUI that shouldn't be any more of an issue than it is for the
> TPIDRs which we don't have a barrier for.  It's possible I'm
> misunderstanding though.

We should be alright without since we'll eventually have an ERET to EL0.

> > > +	/*
> > > +	 * Ensure that GCS changes are observable by/from other PEs in
> > > +	 * case of migration.
> > > +	 */
> > > +	if (task_gcs_el0_enabled(current) || task_gcs_el0_enabled(next))
> > > +		gcsb_dsync();
> 
> > What's this barrier for? The spec (at least the version I have) only
> > talks about accesses, nothing to do with the registers that we context
> > switch here.
> 
> Right, it's for the GCS memory rather than the registers.  I'm fairly
> sure it's excessive but but was erring on the side of caution until I
> have convinced myself that the interactions between GCS barriers and
> regular barriers were doing the right thing, until we have physical
> implementations to contend with I'd guess the practical impact will be
> minimal.

Well, I'd say either we are clear about why it's (not) needed or we ask
the architects to clarify the spec. I haven't checked your latest
series but in principle I don't like adding barriers just because we are
not sure they are needed (and I don't think having hardware eventually
changes this).
Mark Brown Aug. 22, 2023, 5:01 p.m. UTC | #4
On Tue, Aug 22, 2023 at 05:34:38PM +0100, Catalin Marinas wrote:
> On Wed, Aug 16, 2023 at 07:15:53PM +0100, Mark Brown wrote:

> > Right, it's for the GCS memory rather than the registers.  I'm fairly
> > sure it's excessive but but was erring on the side of caution until I
> > have convinced myself that the interactions between GCS barriers and
> > regular barriers were doing the right thing, until we have physical
> > implementations to contend with I'd guess the practical impact will be
> > minimal.

> Well, I'd say either we are clear about why it's (not) needed or we ask
> the architects to clarify the spec. I haven't checked your latest
> series but in principle I don't like adding barriers just because we are
> not sure they are needed (and I don't think having hardware eventually
> changes this).

I should probably also mention that another part of my thinking was that
when we implement GCS for EL1 we'll need to ensure that everything is
synced during the pivot of the EL1 GCS (each EL needs an independent
GCS).  We won't be able to rely on having an ERET there so it's going to
have more stringent requirements, I was partly punting to think things
through fully there.
diff mbox series

Patch

diff --git a/arch/arm64/include/asm/gcs.h b/arch/arm64/include/asm/gcs.h
index 7c5e95218db6..04594ef59dad 100644
--- a/arch/arm64/include/asm/gcs.h
+++ b/arch/arm64/include/asm/gcs.h
@@ -48,4 +48,28 @@  static inline u64 gcsss2(void)
 	return Xt;
 }
 
+#ifdef CONFIG_ARM64_GCS
+
+static inline bool task_gcs_el0_enabled(struct task_struct *task)
+{
+	return current->thread.gcs_el0_mode & PR_SHADOW_STACK_ENABLE;
+}
+
+void gcs_set_el0_mode(struct task_struct *task);
+void gcs_free(struct task_struct *task);
+void gcs_preserve_current_state(void);
+
+#else
+
+static inline bool task_gcs_el0_enabled(struct task_struct *task)
+{
+	return false;
+}
+
+static inline void gcs_set_el0_mode(struct task_struct *task) { }
+static inline void gcs_free(struct task_struct *task) { }
+static inline void gcs_preserve_current_state(void) { }
+
+#endif
+
 #endif
diff --git a/arch/arm64/include/asm/processor.h b/arch/arm64/include/asm/processor.h
index 3918f2a67970..f1551228a143 100644
--- a/arch/arm64/include/asm/processor.h
+++ b/arch/arm64/include/asm/processor.h
@@ -179,6 +179,12 @@  struct thread_struct {
 	u64			sctlr_user;
 	u64			svcr;
 	u64			tpidr2_el0;
+#ifdef CONFIG_ARM64_GCS
+	unsigned int		gcs_el0_mode;
+	u64			gcspr_el0;
+	u64			gcs_base;
+	u64			gcs_size;
+#endif
 };
 
 static inline unsigned int thread_get_vl(struct thread_struct *thread,
diff --git a/arch/arm64/kernel/process.c b/arch/arm64/kernel/process.c
index 0fcc4eb1a7ab..b8a42471aea3 100644
--- a/arch/arm64/kernel/process.c
+++ b/arch/arm64/kernel/process.c
@@ -48,6 +48,7 @@ 
 #include <asm/cacheflush.h>
 #include <asm/exec.h>
 #include <asm/fpsimd.h>
+#include <asm/gcs.h>
 #include <asm/mmu_context.h>
 #include <asm/mte.h>
 #include <asm/processor.h>
@@ -271,12 +272,31 @@  static void flush_tagged_addr_state(void)
 		clear_thread_flag(TIF_TAGGED_ADDR);
 }
 
+#ifdef CONFIG_ARM64_GCS
+
+static void flush_gcs(void)
+{
+	if (system_supports_gcs()) {
+		gcs_free(current);
+		current->thread.gcs_el0_mode = 0;
+		write_sysreg_s(0, SYS_GCSCRE0_EL1);
+		write_sysreg_s(0, SYS_GCSPR_EL0);
+	}
+}
+
+#else
+
+static void flush_gcs(void) { }
+
+#endif
+
 void flush_thread(void)
 {
 	fpsimd_flush_thread();
 	tls_thread_flush();
 	flush_ptrace_hw_breakpoint(current);
 	flush_tagged_addr_state();
+	flush_gcs();
 }
 
 void arch_release_task_struct(struct task_struct *tsk)
@@ -474,6 +494,40 @@  static void entry_task_switch(struct task_struct *next)
 	__this_cpu_write(__entry_task, next);
 }
 
+#ifdef CONFIG_ARM64_GCS
+
+void gcs_preserve_current_state(void)
+{
+	if (task_gcs_el0_enabled(current))
+		current->thread.gcspr_el0 = read_sysreg_s(SYS_GCSPR_EL0);
+}
+
+static void gcs_thread_switch(struct task_struct *next)
+{
+	if (!system_supports_gcs())
+		return;
+
+	gcs_preserve_current_state();
+
+	/*
+	 * Ensure that GCS changes are observable by/from other PEs in
+	 * case of migration.
+	 */
+	if (task_gcs_el0_enabled(current) || task_gcs_el0_enabled(next))
+		gcsb_dsync();
+
+	gcs_set_el0_mode(next);
+	write_sysreg_s(next->thread.gcspr_el0, SYS_GCSPR_EL0);
+}
+
+#else
+
+static void gcs_thread_switch(struct task_struct *next)
+{
+}
+
+#endif
+
 /*
  * ARM erratum 1418040 handling, affecting the 32bit view of CNTVCT.
  * Ensure access is disabled when switching to a 32bit task, ensure
@@ -533,6 +587,7 @@  struct task_struct *__switch_to(struct task_struct *prev,
 	ssbs_thread_switch(next);
 	erratum_1418040_thread_switch(next);
 	ptrauth_thread_switch_user(next);
+	gcs_thread_switch(next);
 
 	/*
 	 * Complete any pending TLB or cache maintenance on this CPU in case
diff --git a/arch/arm64/mm/Makefile b/arch/arm64/mm/Makefile
index dbd1bc95967d..4e7cb2f02999 100644
--- a/arch/arm64/mm/Makefile
+++ b/arch/arm64/mm/Makefile
@@ -10,6 +10,7 @@  obj-$(CONFIG_TRANS_TABLE)	+= trans_pgd.o
 obj-$(CONFIG_TRANS_TABLE)	+= trans_pgd-asm.o
 obj-$(CONFIG_DEBUG_VIRTUAL)	+= physaddr.o
 obj-$(CONFIG_ARM64_MTE)		+= mteswap.o
+obj-$(CONFIG_ARM64_GCS)		+= gcs.o
 KASAN_SANITIZE_physaddr.o	+= n
 
 obj-$(CONFIG_KASAN)		+= kasan_init.o
diff --git a/arch/arm64/mm/gcs.c b/arch/arm64/mm/gcs.c
new file mode 100644
index 000000000000..b0a67efc522b
--- /dev/null
+++ b/arch/arm64/mm/gcs.c
@@ -0,0 +1,39 @@ 
+// SPDX-License-Identifier: GPL-2.0-only
+
+#include <linux/mm.h>
+#include <linux/mman.h>
+#include <linux/syscalls.h>
+#include <linux/types.h>
+
+#include <asm/cpufeature.h>
+#include <asm/page.h>
+
+/*
+ * Apply the GCS mode configured for the specified task to the
+ * hardware.
+ */
+void gcs_set_el0_mode(struct task_struct *task)
+{
+	u64 gcscre0_el1 = GCSCRE0_EL1_nTR;
+
+	if (task->thread.gcs_el0_mode & PR_SHADOW_STACK_ENABLE)
+		gcscre0_el1 |= GCSCRE0_EL1_RVCHKEN | GCSCRE0_EL1_PCRSEL;
+
+	if (task->thread.gcs_el0_mode & PR_SHADOW_STACK_WRITE)
+		gcscre0_el1 |= GCSCRE0_EL1_STREn;
+
+	if (task->thread.gcs_el0_mode & PR_SHADOW_STACK_PUSH)
+		gcscre0_el1 |= GCSCRE0_EL1_PUSHMEn;
+
+	write_sysreg_s(gcscre0_el1, SYS_GCSCRE0_EL1);
+}
+
+void gcs_free(struct task_struct *task)
+{
+	if (task->thread.gcs_base)
+		vm_munmap(task->thread.gcs_base, task->thread.gcs_size);
+
+	task->thread.gcspr_el0 = 0;
+	task->thread.gcs_base = 0;
+	task->thread.gcs_size = 0;
+}