diff mbox series

[v2] crypto: serpent - mark __serpent_setkey_sbox noinline

Message ID 20190618111953.3183723-1-arnd@arndb.de
State New
Headers show
Series [v2] crypto: serpent - mark __serpent_setkey_sbox noinline | expand

Commit Message

Arnd Bergmann June 18, 2019, 11:19 a.m. UTC
The same bug that gcc hit in the past is apparently now showing
up with clang, which decides to inline __serpent_setkey_sbox:

crypto/serpent_generic.c:268:5: error: stack frame size of 2112 bytes in function '__serpent_setkey' [-Werror,-Wframe-larger-than=]

Marking it 'noinline' reduces the stack usage from 2112 bytes to
192 and 96 bytes, respectively, and seems to generate more
useful object code.

Fixes: c871c10e4ea7 ("crypto: serpent - improve __serpent_setkey with UBSAN")
Signed-off-by: Arnd Bergmann <arnd@arndb.de>

---
v2: style improvements suggested by Eric Biggers
---
 crypto/serpent_generic.c | 8 +++++++-
 1 file changed, 7 insertions(+), 1 deletion(-)

-- 
2.20.0

Comments

Eric Biggers June 18, 2019, 6:05 p.m. UTC | #1
On Tue, Jun 18, 2019 at 01:19:42PM +0200, Arnd Bergmann wrote:
> The same bug that gcc hit in the past is apparently now showing

> up with clang, which decides to inline __serpent_setkey_sbox:

> 

> crypto/serpent_generic.c:268:5: error: stack frame size of 2112 bytes in function '__serpent_setkey' [-Werror,-Wframe-larger-than=]

> 

> Marking it 'noinline' reduces the stack usage from 2112 bytes to

> 192 and 96 bytes, respectively, and seems to generate more

> useful object code.

> 

> Fixes: c871c10e4ea7 ("crypto: serpent - improve __serpent_setkey with UBSAN")

> Signed-off-by: Arnd Bergmann <arnd@arndb.de>

> ---

> v2: style improvements suggested by Eric Biggers

> ---

>  crypto/serpent_generic.c | 8 +++++++-

>  1 file changed, 7 insertions(+), 1 deletion(-)

> 

> diff --git a/crypto/serpent_generic.c b/crypto/serpent_generic.c

> index e57757904677..56fa665a4f01 100644

> --- a/crypto/serpent_generic.c

> +++ b/crypto/serpent_generic.c

> @@ -225,7 +225,13 @@

>  	x4 ^= x2;					\

>  	})

>  

> -static void __serpent_setkey_sbox(u32 r0, u32 r1, u32 r2, u32 r3, u32 r4, u32 *k)

> +/*

> + * both gcc and clang have misoptimized this function in the past,

> + * producing horrible object code from spilling temporary variables

> + * on the stack. Forcing this part out of line avoids that.

> + */

> +static noinline void __serpent_setkey_sbox(u32 r0, u32 r1, u32 r2,

> +					   u32 r3, u32 r4, u32 *k)

>  {

>  	k += 100;

>  	S3(r3, r4, r0, r1, r2); store_and_load_keys(r1, r2, r4, r3, 28, 24);

> -- 

> 2.20.0

> 


Reviewed-by: Eric Biggers <ebiggers@kernel.org>


- Eric
Herbert Xu June 28, 2019, 4:19 a.m. UTC | #2
On Tue, Jun 18, 2019 at 01:19:42PM +0200, Arnd Bergmann wrote:
> The same bug that gcc hit in the past is apparently now showing

> up with clang, which decides to inline __serpent_setkey_sbox:

> 

> crypto/serpent_generic.c:268:5: error: stack frame size of 2112 bytes in function '__serpent_setkey' [-Werror,-Wframe-larger-than=]

> 

> Marking it 'noinline' reduces the stack usage from 2112 bytes to

> 192 and 96 bytes, respectively, and seems to generate more

> useful object code.

> 

> Fixes: c871c10e4ea7 ("crypto: serpent - improve __serpent_setkey with UBSAN")

> Signed-off-by: Arnd Bergmann <arnd@arndb.de>

> ---

> v2: style improvements suggested by Eric Biggers

> ---

>  crypto/serpent_generic.c | 8 +++++++-

>  1 file changed, 7 insertions(+), 1 deletion(-)


Patch applied.  Thanks.
-- 
Email: Herbert Xu <herbert@gondor.apana.org.au>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt
Eric Biggers June 28, 2019, 5:04 p.m. UTC | #3
On Fri, Jun 28, 2019 at 12:19:12PM +0800, Herbert Xu wrote:
> On Tue, Jun 18, 2019 at 01:19:42PM +0200, Arnd Bergmann wrote:

> > The same bug that gcc hit in the past is apparently now showing

> > up with clang, which decides to inline __serpent_setkey_sbox:

> > 

> > crypto/serpent_generic.c:268:5: error: stack frame size of 2112 bytes in function '__serpent_setkey' [-Werror,-Wframe-larger-than=]

> > 

> > Marking it 'noinline' reduces the stack usage from 2112 bytes to

> > 192 and 96 bytes, respectively, and seems to generate more

> > useful object code.

> > 

> > Fixes: c871c10e4ea7 ("crypto: serpent - improve __serpent_setkey with UBSAN")

> > Signed-off-by: Arnd Bergmann <arnd@arndb.de>

> > ---

> > v2: style improvements suggested by Eric Biggers

> > ---

> >  crypto/serpent_generic.c | 8 +++++++-

> >  1 file changed, 7 insertions(+), 1 deletion(-)

> 

> Patch applied.  Thanks.

> -- 


Hi Herbert, seems you forgot to push?  I don't see this in cryptodev.

- Eric
Herbert Xu June 29, 2019, 1:15 a.m. UTC | #4
On Fri, Jun 28, 2019 at 10:04:38AM -0700, Eric Biggers wrote:
>

> Hi Herbert, seems you forgot to push?  I don't see this in cryptodev.


Yes you're right.  It should be there now.

Thanks,
-- 
Email: Herbert Xu <herbert@gondor.apana.org.au>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt
diff mbox series

Patch

diff --git a/crypto/serpent_generic.c b/crypto/serpent_generic.c
index e57757904677..56fa665a4f01 100644
--- a/crypto/serpent_generic.c
+++ b/crypto/serpent_generic.c
@@ -225,7 +225,13 @@ 
 	x4 ^= x2;					\
 	})
 
-static void __serpent_setkey_sbox(u32 r0, u32 r1, u32 r2, u32 r3, u32 r4, u32 *k)
+/*
+ * both gcc and clang have misoptimized this function in the past,
+ * producing horrible object code from spilling temporary variables
+ * on the stack. Forcing this part out of line avoids that.
+ */
+static noinline void __serpent_setkey_sbox(u32 r0, u32 r1, u32 r2,
+					   u32 r3, u32 r4, u32 *k)
 {
 	k += 100;
 	S3(r3, r4, r0, r1, r2); store_and_load_keys(r1, r2, r4, r3, 28, 24);