From patchwork Wed May 11 10:13:43 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 67495 Delivered-To: patch@linaro.org Received: by 10.140.92.199 with SMTP id b65csp139607qge; Wed, 11 May 2016 03:13:53 -0700 (PDT) X-Received: by 10.66.65.235 with SMTP id a11mr3474840pat.155.1462961633653; Wed, 11 May 2016 03:13:53 -0700 (PDT) Return-Path: Received: from ml01.01.org (ml01.01.org. [198.145.21.10]) by mx.google.com with ESMTPS id vw4si8874301pab.180.2016.05.11.03.13.53 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 11 May 2016 03:13:53 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of edk2-devel-bounces@lists.01.org designates 198.145.21.10 as permitted sender) client-ip=198.145.21.10; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@linaro.org; spf=pass (google.com: best guess record for domain of edk2-devel-bounces@lists.01.org designates 198.145.21.10 as permitted sender) smtp.mailfrom=edk2-devel-bounces@lists.01.org; dmarc=fail (p=NONE dis=NONE) header.from=linaro.org Received: from [127.0.0.1] (localhost [IPv6:::1]) by ml01.01.org (Postfix) with ESMTP id 177F61A1F60; Wed, 11 May 2016 03:13:53 -0700 (PDT) X-Original-To: edk2-devel@lists.01.org Delivered-To: edk2-devel@lists.01.org Received: from mail-wm0-x231.google.com (mail-wm0-x231.google.com [IPv6:2a00:1450:400c:c09::231]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ml01.01.org (Postfix) with ESMTPS id DFFFE1A1F5C for ; Wed, 11 May 2016 03:13:51 -0700 (PDT) Received: by mail-wm0-x231.google.com with SMTP id a17so73592750wme.0 for ; Wed, 11 May 2016 03:13:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id; bh=jQn6/JuRqzne3BVtI3iQ16PTuGaq7rUC1w4DNqt/XME=; b=jXaLWSGlOOMvBkRYZiiuljdtjUsD33+g4EQ0W9eqqpba0RUGggzUwJPTWfAPhDTyhb kMg7cSl77lBizTCEqYj6+oM3I2dyLkjzSWOvSni3xzCtHxeG5R7upUkahYv6i9Krcxbx Wze1P4Os8e4MSoL3oLBkTEzJ+Gy8FFlNPtFlI= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:from:to:cc:subject:date:message-id; bh=jQn6/JuRqzne3BVtI3iQ16PTuGaq7rUC1w4DNqt/XME=; b=XSW4S7riGrEnQIVprxMJhhsBsHAm6IQkQ0jMVC9jJFb3tNe1Tnub3kynr/onm0UbFS giRLLdQ2zDweyxDqDYvjBwioydlJG7H9Oa2TH+iLakKI2qDhud6SOzKJ594RGd4BK2nF 8zQF1BbzlgWRvotXEUvUA12u7ibql+vkTHbx+BBztl8or6qFNjTLDW7bb23Ds0ceOIWJ tXurpRY9B88iGsbbhBA+aObvVq8W4hqwrz0aC4KvtjVH2mreYavNTBgg3DUbiKPXF/RU VTwIzXsA3YAXmfBcd6LDEv64VIrTqv5V6GUVI80R6GKpqaMPL1+ZV4sEH8BI78fFKcwc PVvA== X-Gm-Message-State: AOPr4FX+aQMxhCWCC1aO89m6mvZyXEdSzeRpwjtg3GSpf3ZUE2YxjoYL4kId2Z5Tant2jFcj X-Received: by 10.194.104.40 with SMTP id gb8mr2672429wjb.84.1462961630256; Wed, 11 May 2016 03:13:50 -0700 (PDT) Received: from localhost.localdomain ([195.55.142.58]) by smtp.gmail.com with ESMTPSA id on2sm7226164wjc.32.2016.05.11.03.13.48 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Wed, 11 May 2016 03:13:49 -0700 (PDT) From: Ard Biesheuvel To: edk2-devel@lists.01.org, leif.lindholm@linaro.org Date: Wed, 11 May 2016 12:13:43 +0200 Message-Id: <1462961623-15996-1-git-send-email-ard.biesheuvel@linaro.org> X-Mailer: git-send-email 2.7.4 Subject: [edk2] [PATCH v2] ArmPkg/ArmLib: don't invalidate entire I-cache on range operation X-BeenThere: edk2-devel@lists.01.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: EDK II Development List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: Ard Biesheuvel , eugene@hp.com MIME-Version: 1.0 Errors-To: edk2-devel-bounces@lists.01.org Sender: "edk2-devel" Instead of cleaning the data cache to the PoU by virtual address and subsequently invalidating the entire I-cache, invalidate only the range that we just cleaned. This way, we don't invalidate other cachelines unnecessarily. Contributed-under: TianoCore Contribution Agreement 1.0 Signed-off-by: Ard Biesheuvel --- v2: - use correct stride for Icache maintenance by VA - issue an ISB after completing the Icache maintenance ArmPkg/Include/Library/ArmLib.h | 10 +++++++-- ArmPkg/Library/ArmCacheMaintenanceLib/ArmCacheMaintenanceLib.c | 23 +++++++++++++++----- ArmPkg/Library/ArmLib/AArch64/AArch64Support.S | 5 +++++ ArmPkg/Library/ArmLib/ArmV7/ArmV7Support.S | 5 +++++ ArmPkg/Library/ArmLib/ArmV7/ArmV7Support.asm | 6 +++++ 5 files changed, 41 insertions(+), 8 deletions(-) -- 2.7.4 _______________________________________________ edk2-devel mailing list edk2-devel@lists.01.org https://lists.01.org/mailman/listinfo/edk2-devel Reviewed-by: Leif Lindholm diff --git a/ArmPkg/Include/Library/ArmLib.h b/ArmPkg/Include/Library/ArmLib.h index 1689f0072db6..4608b0cccccc 100644 --- a/ArmPkg/Include/Library/ArmLib.h +++ b/ArmPkg/Include/Library/ArmLib.h @@ -183,13 +183,19 @@ ArmInvalidateDataCacheEntryByMVA ( VOID EFIAPI -ArmCleanDataCacheEntryToPoUByMVA( +ArmCleanDataCacheEntryToPoUByMVA ( IN UINTN Address ); VOID EFIAPI -ArmCleanDataCacheEntryByMVA( +ArmInvalidateInstructionCacheEntryToPoUByMVA ( + IN UINTN Address + ); + +VOID +EFIAPI +ArmCleanDataCacheEntryByMVA ( IN UINTN Address ); diff --git a/ArmPkg/Library/ArmCacheMaintenanceLib/ArmCacheMaintenanceLib.c b/ArmPkg/Library/ArmCacheMaintenanceLib/ArmCacheMaintenanceLib.c index 1045f9068f4d..f7667b8552d2 100644 --- a/ArmPkg/Library/ArmCacheMaintenanceLib/ArmCacheMaintenanceLib.c +++ b/ArmPkg/Library/ArmCacheMaintenanceLib/ArmCacheMaintenanceLib.c @@ -17,11 +17,13 @@ #include #include +STATIC VOID CacheRangeOperation ( IN VOID *Start, IN UINTN Length, - IN LINE_OPERATION LineOperation + IN LINE_OPERATION LineOperation, + IN UINTN LineLength ) { UINTN ArmCacheLineLength = ArmDataCacheLineLength(); @@ -64,8 +66,14 @@ InvalidateInstructionCacheRange ( IN UINTN Length ) { - CacheRangeOperation (Address, Length, ArmCleanDataCacheEntryToPoUByMVA); - ArmInvalidateInstructionCache (); + CacheRangeOperation (Address, Length, ArmCleanDataCacheEntryToPoUByMVA, + ArmDataCacheLineLength ()); + CacheRangeOperation (Address, Length, + ArmInvalidateInstructionCacheEntryToPoUByMVA, + ArmInstructionCacheLineLength ()); + + ArmInstructionSynchronizationBarrier (); + return Address; } @@ -85,7 +93,8 @@ WriteBackInvalidateDataCacheRange ( IN UINTN Length ) { - CacheRangeOperation(Address, Length, ArmCleanInvalidateDataCacheEntryByMVA); + CacheRangeOperation(Address, Length, ArmCleanInvalidateDataCacheEntryByMVA, + ArmDataCacheLineLength ()); return Address; } @@ -105,7 +114,8 @@ WriteBackDataCacheRange ( IN UINTN Length ) { - CacheRangeOperation(Address, Length, ArmCleanDataCacheEntryByMVA); + CacheRangeOperation(Address, Length, ArmCleanDataCacheEntryByMVA, + ArmDataCacheLineLength ()); return Address; } @@ -116,6 +126,7 @@ InvalidateDataCacheRange ( IN UINTN Length ) { - CacheRangeOperation(Address, Length, ArmInvalidateDataCacheEntryByMVA); + CacheRangeOperation(Address, Length, ArmInvalidateDataCacheEntryByMVA, + ArmDataCacheLineLength ()); return Address; } diff --git a/ArmPkg/Library/ArmLib/AArch64/AArch64Support.S b/ArmPkg/Library/ArmLib/AArch64/AArch64Support.S index 43f7a795acec..9441f47e30ba 100644 --- a/ArmPkg/Library/ArmLib/AArch64/AArch64Support.S +++ b/ArmPkg/Library/ArmLib/AArch64/AArch64Support.S @@ -23,6 +23,7 @@ GCC_ASM_EXPORT (ArmInvalidateInstructionCache) GCC_ASM_EXPORT (ArmInvalidateDataCacheEntryByMVA) GCC_ASM_EXPORT (ArmCleanDataCacheEntryByMVA) GCC_ASM_EXPORT (ArmCleanDataCacheEntryToPoUByMVA) +GCC_ASM_EXPORT (ArmInvalidateInstructionCacheEntryToPoUByMVA) GCC_ASM_EXPORT (ArmCleanInvalidateDataCacheEntryByMVA) GCC_ASM_EXPORT (ArmInvalidateDataCacheEntryBySetWay) GCC_ASM_EXPORT (ArmCleanDataCacheEntryBySetWay) @@ -80,6 +81,10 @@ ASM_PFX(ArmCleanDataCacheEntryToPoUByMVA): dc cvau, x0 // Clean single data cache line to PoU ret +ASM_PFX(ArmInvalidateInstructionCacheEntryToPoUByMVA): + ic ivau, x0 // Invalidate single instruction cache line to PoU + ret + ASM_PFX(ArmCleanInvalidateDataCacheEntryByMVA): dc civac, x0 // Clean and invalidate single data cache line diff --git a/ArmPkg/Library/ArmLib/ArmV7/ArmV7Support.S b/ArmPkg/Library/ArmLib/ArmV7/ArmV7Support.S index 50c760f335de..c765032c9e4d 100644 --- a/ArmPkg/Library/ArmLib/ArmV7/ArmV7Support.S +++ b/ArmPkg/Library/ArmLib/ArmV7/ArmV7Support.S @@ -18,6 +18,7 @@ GCC_ASM_EXPORT (ArmInvalidateInstructionCache) GCC_ASM_EXPORT (ArmInvalidateDataCacheEntryByMVA) +GCC_ASM_EXPORT (ArmInvalidateInstructionCacheEntryToPoUByMVA) GCC_ASM_EXPORT (ArmCleanDataCacheEntryByMVA) GCC_ASM_EXPORT (ArmCleanDataCacheEntryToPoUByMVA) GCC_ASM_EXPORT (ArmCleanInvalidateDataCacheEntryByMVA) @@ -74,6 +75,10 @@ ASM_PFX(ArmCleanDataCacheEntryToPoUByMVA): mcr p15, 0, r0, c7, c11, 1 @clean single data cache line to PoU bx lr +ASM_PFX(ArmInvalidateInstructionCacheEntryToPoUByMVA): + mcr p15, 0, r0, c7, c5, 1 @Invalidate single instruction cache line to PoU + mcr p15, 0, r0, c7, c5, 7 @Invalidate branch predictor + bx lr ASM_PFX(ArmCleanInvalidateDataCacheEntryByMVA): mcr p15, 0, r0, c7, c14, 1 @clean and invalidate single data cache line diff --git a/ArmPkg/Library/ArmLib/ArmV7/ArmV7Support.asm b/ArmPkg/Library/ArmLib/ArmV7/ArmV7Support.asm index a460bd2da7a9..2363ee457632 100644 --- a/ArmPkg/Library/ArmLib/ArmV7/ArmV7Support.asm +++ b/ArmPkg/Library/ArmLib/ArmV7/ArmV7Support.asm @@ -34,6 +34,12 @@ CTRL_I_BIT EQU (1 << 12) bx lr + RVCT_ASM_EXPORT ArmInvalidateInstructionCacheEntryToPoUByMVA + mcr p15, 0, r0, c7, c5, 1 ; invalidate single instruction cache line to PoU + mcr p15, 0, r0, c7, c5, 7 ; invalidate branch predictor + bx lr + + RVCT_ASM_EXPORT ArmCleanDataCacheEntryToPoUByMVA mcr p15, 0, r0, c7, c11, 1 ; clean single data cache line to PoU bx lr