From patchwork Wed Dec 6 19:43:37 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 120895 Delivered-To: patch@linaro.org Received: by 10.140.22.227 with SMTP id 90csp7469002qgn; Wed, 6 Dec 2017 11:44:42 -0800 (PST) X-Google-Smtp-Source: AGs4zMb23F2pgq9I4vUWJKXi8XkcA6GowpneEVuCWy89fAzh5eh2tB/ySVWErweXQJJ7RzlXLndH X-Received: by 10.101.99.140 with SMTP id h12mr21868859pgv.80.1512589482813; Wed, 06 Dec 2017 11:44:42 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1512589482; cv=none; d=google.com; s=arc-20160816; b=zvT3v7Ol104VMp37NoZ0h4qNoymStUhlevP7MIo9va6wb9reLrkrZGO4uLqGIZkrFD QQrLwbkbjlvHcxutoYl1FY8FbNehJsHGl13OZE92VAHLXlMGe0+P90+qW6dSSAlQms37 6eAu5YMhsSVykJYJOSJdYjHQ+SDBnSQyEZEsCsaMvESyE9zRJbh5EpPhTIAIAdSgAeP4 K8yuTpfZlpl/PPlJo+S7OyIT47qCj6PWnEWjH/Zf9TmhlFm1lrXbL3lHtASWUl85d4k2 0CUTn4aXCiGO/QhrUeBO2KNbuhx/M90MPfRze3pn0Ze+GkBkbvHoYrswCQLtsqxyY5rZ wHXw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from:dkim-signature:arc-authentication-results; bh=nktoeluwbKnt4qPDY2lQ8CC4zE2KSWsLD5ZbGr2gOFI=; b=C+eWrQqwpgInXqBHgljqZZmnzsrB+7+W0xfineRoLI6fM24A8vwMCD2uFyC/QZ0ufo +xZGUGyB1DNqX581unYXeVKPB3FsKXqQ9KKZieVJZbdV9z02fuGxqmrp/8X4a1AAWrSB 4BQfYlK02A+UhWcQJWctl4h2/9Vn/GXNaCjiyuW1J4pJ8L3/njkoMZgtmTKEGywbEmKc lGYNov0R90KpabAqQBgbEg78tvHrEdU95qxcAdm0F4XG1Z5DUTxUGwyD3sYt9mk3Slek emcFvScMQXAUoICbnn5GwX96HWPCJBDNST+8gnoyXjaEgsgKInGRQZUKHMTXMK6rqTW4 yk9Q== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=TLIR2OK7; spf=pass (google.com: best guess record for domain of linux-crypto-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-crypto-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id i16si2382524pgv.496.2017.12.06.11.44.42; Wed, 06 Dec 2017 11:44:42 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-crypto-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=TLIR2OK7; spf=pass (google.com: best guess record for domain of linux-crypto-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-crypto-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752413AbdLFTol (ORCPT + 1 other); Wed, 6 Dec 2017 14:44:41 -0500 Received: from mail-wm0-f68.google.com ([74.125.82.68]:36664 "EHLO mail-wm0-f68.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752176AbdLFTo3 (ORCPT ); Wed, 6 Dec 2017 14:44:29 -0500 Received: by mail-wm0-f68.google.com with SMTP id b76so9175704wmg.1 for ; Wed, 06 Dec 2017 11:44:28 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=nktoeluwbKnt4qPDY2lQ8CC4zE2KSWsLD5ZbGr2gOFI=; b=TLIR2OK7SqQNGNB26u40vVnifqhEgu6OFAPQ7uOkXIABMWuVPc4N7O1ZxVbcwAsiQv IbQFgxYL+MhURDFqkFvM82/dJZQmSP4swialookiLotqUUVqR5ZTTn5cDbSbEhMIPCOz RMebwo4VzKFo0zrtZq8xx4+sHDQQb8YOd2bAY= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=nktoeluwbKnt4qPDY2lQ8CC4zE2KSWsLD5ZbGr2gOFI=; b=UVzVFFt/WDCM99BNJ2wB8Ok1puzvWUVjruO1CUiaHWDwK70Ug5DTy3zNAS9qzsuJqp oQvBV3wAtH/t6uV64XbU3GM8X4bfW5pqn7yZvMn9bNn5PEtGrpbcbDkTZk3uqOTIG4gE r11BWZACO1IHw9sfj29RlfBegGKPLva64+VV5dz++UJwXJHSvAiqRGsv4vah90yhBG4N 1+cCOwOw2V2wen5q01KScWnFL35fyPIIcLLTsdwOctE85/1rzrwWoucfjuoZGcb5/Ww7 VKQM+PRr80QXPh0fssUYlDX2j4Xy799jRJ2CV1OEd6+DWRjZiB2ZZPc5O6A2BvEgT5zL r/FQ== X-Gm-Message-State: AKGB3mLXTFvSyByD3I5PT3ouMM9uXEw+dibzIKudrkxlo/XPrgMxsDw0 0EbKKYv3bQw40hARWWXX5LlFzTX2BCM= X-Received: by 10.28.54.3 with SMTP id d3mr15363015wma.79.1512589467460; Wed, 06 Dec 2017 11:44:27 -0800 (PST) Received: from localhost.localdomain ([105.150.171.234]) by smtp.gmail.com with ESMTPSA id b66sm3596594wmh.32.2017.12.06.11.44.24 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 06 Dec 2017 11:44:26 -0800 (PST) From: Ard Biesheuvel To: linux-crypto@vger.kernel.org Cc: herbert@gondor.apana.org.au, linux-arm-kernel@lists.infradead.org, Ard Biesheuvel , Dave Martin , Russell King - ARM Linux , Sebastian Andrzej Siewior , Mark Rutland , linux-rt-users@vger.kernel.org, Peter Zijlstra , Catalin Marinas , Will Deacon , Steven Rostedt , Thomas Gleixner Subject: [PATCH v3 11/20] arm64: assembler: add macros to conditionally yield the NEON under PREEMPT Date: Wed, 6 Dec 2017 19:43:37 +0000 Message-Id: <20171206194346.24393-12-ard.biesheuvel@linaro.org> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20171206194346.24393-1-ard.biesheuvel@linaro.org> References: <20171206194346.24393-1-ard.biesheuvel@linaro.org> Sender: linux-crypto-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-crypto@vger.kernel.org Add support macros to conditionally yield the NEON (and thus the CPU) that may be called from the assembler code. In some cases, yielding the NEON involves saving and restoring a non trivial amount of context (especially in the CRC folding algorithms), and so the macro is split into three, and the code in between is only executed when the yield path is taken, allowing the context to be preserved. The third macro takes an optional label argument that marks the resume path after a yield has been performed. Signed-off-by: Ard Biesheuvel --- arch/arm64/include/asm/assembler.h | 51 ++++++++++++++++++++ 1 file changed, 51 insertions(+) -- 2.11.0 diff --git a/arch/arm64/include/asm/assembler.h b/arch/arm64/include/asm/assembler.h index 5f61487e9f93..c54e408fd5a7 100644 --- a/arch/arm64/include/asm/assembler.h +++ b/arch/arm64/include/asm/assembler.h @@ -572,4 +572,55 @@ alternative_else_nop_endif #endif .endm +/* + * Check whether to yield to another runnable task from kernel mode NEON code + * (which runs with preemption disabled). + * + * if_will_cond_yield_neon + * // pre-yield patchup code + * do_cond_yield_neon + * // post-yield patchup code + * endif_yield_neon + * + * - Check whether the preempt count is exactly 1, in which case disabling + * preemption once will make the task preemptible. If this is not the case, + * yielding is pointless. + * - Check whether TIF_NEED_RESCHED is set, and if so, disable and re-enable + * kernel mode NEON (which will trigger a reschedule), and branch to the + * yield fixup code. + * + * This macro sequence clobbers x0, x1 and the flags register unconditionally, + * and may clobber x2 .. x18 if the yield path is taken. + */ + + .macro cond_yield_neon, lbl + if_will_cond_yield_neon + do_cond_yield_neon + endif_yield_neon \lbl + .endm + + .macro if_will_cond_yield_neon +#ifdef CONFIG_PREEMPT + get_thread_info x0 + ldr w1, [x0, #TSK_TI_PREEMPT] + ldr x0, [x0, #TSK_TI_FLAGS] + cmp w1, #1 // == PREEMPT_OFFSET + csel x0, x0, xzr, eq + tbnz x0, #TIF_NEED_RESCHED, 5555f // needs rescheduling? +#endif + .subsection 1 +5555: + .endm + + .macro do_cond_yield_neon + bl kernel_neon_end + bl kernel_neon_begin + .endm + + .macro endif_yield_neon, lbl=6666f + b \lbl + .previous +6666: + .endm + #endif /* __ASM_ASSEMBLER_H */