From patchwork Thu Feb 19 17:25:16 2015 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 44848 Return-Path: X-Original-To: linaro@patches.linaro.org Delivered-To: linaro@patches.linaro.org Received: from mail-wg0-f69.google.com (mail-wg0-f69.google.com [74.125.82.69]) by ip-10-151-82-157.ec2.internal (Postfix) with ESMTPS id CCFDE21554 for ; Thu, 19 Feb 2015 17:25:32 +0000 (UTC) Received: by mail-wg0-f69.google.com with SMTP id k14sf5991138wgh.0 for ; Thu, 19 Feb 2015 09:25:32 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:delivered-to:from:to:cc:subject :date:message-id:sender:precedence:list-id:x-original-sender :x-original-authentication-results:mailing-list:list-post:list-help :list-archive:list-unsubscribe; bh=kBCF1nmlmq0UHs6bj0CxgA0UPrS0D2vD2odkGpM0Jvk=; b=a4awHwUSOTzYBuG8h1ySeFqUqQjZKMw/6RbF5K6JidhqfjG+4Ty+8sVrZgNNMHwYGq Hx6sKDdNHUJl5bPpCXLbGWSLrvsaTZY1CxJG6ZNU98NYk5MEIpEqFx5PUyFrFKq/SzBr meBbGLHBWKhyDKzCcliyUSSRZ0usNiWstn2I2/OBLnguQyJMXdnM6Cntoq1Gdulj2G2D NzPJJZZ8+1IhrL8jApyUBPuSXbJz39aD5wuVEX1idqB8LkDyTOO2ueEsKglRCdmaM1Ls tzrCc/FMBbfgLMIMqie1QaMx+3caozj8yONxsZAzCNPKNWJn+1eR3ZLnu8BMvBncL4Pn 5aqQ== X-Gm-Message-State: ALoCoQltdbtViEyxgT5g7Y9zia6tXBAgKAsKNBU3Yx22Sc5zkouojsDK0IUViNuVuTd+gpM68EWe X-Received: by 10.112.45.197 with SMTP id p5mr767440lbm.18.1424366732074; Thu, 19 Feb 2015 09:25:32 -0800 (PST) MIME-Version: 1.0 X-BeenThere: patchwork-forward@linaro.org Received: by 10.152.26.165 with SMTP id m5ls188558lag.31.gmail; Thu, 19 Feb 2015 09:25:31 -0800 (PST) X-Received: by 10.152.23.233 with SMTP id p9mr4816495laf.123.1424366731810; Thu, 19 Feb 2015 09:25:31 -0800 (PST) Received: from mail-la0-f45.google.com (mail-la0-f45.google.com. [209.85.215.45]) by mx.google.com with ESMTPS id jc11si16145463lac.15.2015.02.19.09.25.31 for (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 19 Feb 2015 09:25:31 -0800 (PST) Received-SPF: pass (google.com: domain of patch+caf_=patchwork-forward=linaro.org@linaro.org designates 209.85.215.45 as permitted sender) client-ip=209.85.215.45; Received: by labms9 with SMTP id ms9so961516lab.10 for ; Thu, 19 Feb 2015 09:25:31 -0800 (PST) X-Received: by 10.112.42.225 with SMTP id r1mr1450841lbl.72.1424366731640; Thu, 19 Feb 2015 09:25:31 -0800 (PST) X-Forwarded-To: patchwork-forward@linaro.org X-Forwarded-For: patch@linaro.org patchwork-forward@linaro.org Delivered-To: patch@linaro.org Received: by 10.112.35.133 with SMTP id h5csp634260lbj; Thu, 19 Feb 2015 09:25:30 -0800 (PST) X-Received: by 10.70.133.168 with SMTP id pd8mr9533518pdb.122.1424366729767; Thu, 19 Feb 2015 09:25:29 -0800 (PST) Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id g13si14691753pat.64.2015.02.19.09.25.29 for ; Thu, 19 Feb 2015 09:25:29 -0800 (PST) Received-SPF: none (google.com: linux-crypto-owner@vger.kernel.org does not designate permitted sender hosts) client-ip=209.132.180.67; Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752270AbbBSRZ1 (ORCPT ); Thu, 19 Feb 2015 12:25:27 -0500 Received: from mail-wg0-f54.google.com ([74.125.82.54]:64136 "EHLO mail-wg0-f54.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752018AbbBSRZ1 (ORCPT ); Thu, 19 Feb 2015 12:25:27 -0500 Received: by mail-wg0-f54.google.com with SMTP id y19so8492627wgg.13 for ; Thu, 19 Feb 2015 09:25:25 -0800 (PST) X-Received: by 10.194.118.198 with SMTP id ko6mr10769004wjb.47.1424366725677; Thu, 19 Feb 2015 09:25:25 -0800 (PST) Received: from ards-macbook-pro.local (237.102.108.93.rev.vodafone.pt. [93.108.102.237]) by mx.google.com with ESMTPSA id ub1sm38285679wjc.43.2015.02.19.09.25.22 (version=TLSv1.1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Thu, 19 Feb 2015 09:25:24 -0800 (PST) From: Ard Biesheuvel To: will.deacon@arm.com, linux-arm-kernel@lists.infradead.org Cc: steve.capper@linaro.org, herbert@gondor.apana.org.au, linux-crypto@vger.kernel.org, Ard Biesheuvel Subject: [PATCH] arm64: crypto: increase AES interleave to 4x Date: Thu, 19 Feb 2015 17:25:16 +0000 Message-Id: <1424366716-30439-1-git-send-email-ard.biesheuvel@linaro.org> X-Mailer: git-send-email 1.8.3.2 Sender: linux-crypto-owner@vger.kernel.org Precedence: list List-ID: X-Mailing-List: linux-crypto@vger.kernel.org X-Removed-Original-Auth: Dkim didn't pass. X-Original-Sender: ard.biesheuvel@linaro.org X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of patch+caf_=patchwork-forward=linaro.org@linaro.org designates 209.85.215.45 as permitted sender) smtp.mail=patch+caf_=patchwork-forward=linaro.org@linaro.org Mailing-list: list patchwork-forward@linaro.org; contact patchwork-forward+owners@linaro.org X-Google-Group-Id: 836684582541 List-Post: , List-Help: , List-Archive: List-Unsubscribe: , This patch increases the interleave factor for parallel AES modes to 4x. This improves performance on Cortex-A57 by ~35%. This is due to the 3-cycle latency of AES instructions on the A57's relatively deep pipeline (compared to Cortex-A53 where the AES instruction latency is only 2 cycles). At the same time, disable inline expansion of the core AES functions, as the performance benefit of this feature is negligible. Measured on AMD Seattle (using tcrypt.ko mode=500 sec=1): Baseline (2x interleave, inline expansion) ------------------------------------------ testing speed of async cbc(aes) (cbc-aes-ce) decryption test 4 (128 bit key, 8192 byte blocks): 95545 operations in 1 seconds test 14 (256 bit key, 8192 byte blocks): 68496 operations in 1 seconds This patch (4x interleave, no inline expansion) ----------------------------------------------- testing speed of async cbc(aes) (cbc-aes-ce) decryption test 4 (128 bit key, 8192 byte blocks): 124735 operations in 1 seconds test 14 (256 bit key, 8192 byte blocks): 92328 operations in 1 seconds Signed-off-by: Ard Biesheuvel --- arch/arm64/crypto/Makefile | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/arch/arm64/crypto/Makefile b/arch/arm64/crypto/Makefile index 5720608c50b1..abb79b3cfcfe 100644 --- a/arch/arm64/crypto/Makefile +++ b/arch/arm64/crypto/Makefile @@ -29,7 +29,7 @@ aes-ce-blk-y := aes-glue-ce.o aes-ce.o obj-$(CONFIG_CRYPTO_AES_ARM64_NEON_BLK) += aes-neon-blk.o aes-neon-blk-y := aes-glue-neon.o aes-neon.o -AFLAGS_aes-ce.o := -DINTERLEAVE=2 -DINTERLEAVE_INLINE +AFLAGS_aes-ce.o := -DINTERLEAVE=4 AFLAGS_aes-neon.o := -DINTERLEAVE=4 CFLAGS_aes-glue-ce.o := -DUSE_V8_CRYPTO_EXTENSIONS