From patchwork Mon Mar 6 17:32:13 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 94928 Delivered-To: patch@linaro.org Received: by 10.140.82.71 with SMTP id g65csp1494350qgd; Mon, 6 Mar 2017 09:32:28 -0800 (PST) X-Received: by 10.99.176.6 with SMTP id h6mr22031027pgf.184.1488821548598; Mon, 06 Mar 2017 09:32:28 -0800 (PST) Return-Path: Received: from ml01.01.org (ml01.01.org. [2001:19d0:306:5::1]) by mx.google.com with ESMTPS id e7si19695066pfa.53.2017.03.06.09.32.28 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 06 Mar 2017 09:32:28 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of edk2-devel-bounces@lists.01.org designates 2001:19d0:306:5::1 as permitted sender) client-ip=2001:19d0:306:5::1; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@linaro.org; spf=pass (google.com: best guess record for domain of edk2-devel-bounces@lists.01.org designates 2001:19d0:306:5::1 as permitted sender) smtp.mailfrom=edk2-devel-bounces@lists.01.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from [127.0.0.1] (localhost [IPv6:::1]) by ml01.01.org (Postfix) with ESMTP id A1BBA80334; Mon, 6 Mar 2017 09:32:26 -0800 (PST) X-Original-To: edk2-devel@lists.01.org Delivered-To: edk2-devel@lists.01.org Received: from mail-wm0-x234.google.com (mail-wm0-x234.google.com [IPv6:2a00:1450:400c:c09::234]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ml01.01.org (Postfix) with ESMTPS id D485B80333 for ; Mon, 6 Mar 2017 09:32:25 -0800 (PST) Received: by mail-wm0-x234.google.com with SMTP id v186so70021591wmd.0 for ; Mon, 06 Mar 2017 09:32:25 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=UFWiLAV8wcx3/Kz+vgcKAFPYoFIQNmFtdLkftb8aSss=; b=RIiPhcieE/u/Du8t/yNyVm5p9Ixba5zsXuLCEazgswg/J8Mm3tOE/Ghqqczu+Rr4MB JFNeGfblIpOHecLkQ8TI8NTmMENv8yG3/KJm6FsbA5In9XXAlQqq2CUtmcxGuG5pf08Q qLcmpfcIJAqYUv5aNlKRBliyk/nRD9FWyNFwA= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=UFWiLAV8wcx3/Kz+vgcKAFPYoFIQNmFtdLkftb8aSss=; b=p13cCpTMFFhUBxKzxz9MPfwDLuAsE/rOC98LqdDM0RBLcAFTU7Ur8H+Rk+bBrQUZcU RMNYh+TcSjJHtKj/5Y166nhnm20obbGaEgvSQ1yu36GzYs15ABbvf0+JQtAn1Y+u9vQs 9pjZV7ON9U572Swzb3WvTzMp3ReKUqdNmUWk3X7VthMnZv6+B1ZhhrK4MbG9O0vfj06K UZ3dkE2VMxi7tRkW7QoNhGNE/OM4X8I+hVZCSha27XV24ySsx/WdzD5dWcmPrPSBtU+N MVZmyPboZBkxnRFY5y5QqApCC9RDNjAZKwH9CNUDh5pYcRocfDHZR/i4aVoHbO6/fq6r U9+w== X-Gm-Message-State: AMke39lFz/z5W8GR1MChrb/zRZIW7bNx6s2WIgkokR3ee9Oa0HdGEJmolsKYAAlbLxhfzVvv X-Received: by 10.28.65.65 with SMTP id o62mr13504140wma.14.1488821544395; Mon, 06 Mar 2017 09:32:24 -0800 (PST) Received: from ards-macbook-pro.c.hoisthospitality.com ([109.74.48.129]) by smtp.gmail.com with ESMTPSA id 136sm15704335wmg.12.2017.03.06.09.32.22 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Mon, 06 Mar 2017 09:32:23 -0800 (PST) From: Ard Biesheuvel To: edk2-devel@lists.01.org, leif.lindholm@linaro.org Date: Mon, 6 Mar 2017 18:32:13 +0100 Message-Id: <1488821535-14795-3-git-send-email-ard.biesheuvel@linaro.org> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1488821535-14795-1-git-send-email-ard.biesheuvel@linaro.org> References: <1488821535-14795-1-git-send-email-ard.biesheuvel@linaro.org> Subject: [edk2] [PATCH v3 2/4] ArmPkg/CpuDxe ARM: avoid unnecessary cache/TLB maintenance X-BeenThere: edk2-devel@lists.01.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: EDK II Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: lersek@redhat.com, Ard Biesheuvel MIME-Version: 1.0 Errors-To: edk2-devel-bounces@lists.01.org Sender: "edk2-devel" Page and section entries in the page tables are updated using the helper ArmUpdateTranslationTableEntry(), which cleans the page table entry to the PoC, and invalidates the TLB entry covering the page described by the entry being updated. Since we may be updating section entries, we might be leaving stale TLB entries at this point (for all pages in the section except the first one), which will be invalidated wholesale at the end of SetMemoryAttributes(). At that point, all caches are cleaned *and* invalidated as well. This cache maintenance is costly and unnecessary. The TLB maintenance is only necessary if we updated any section entries, since any page by page entries that have been updated will have been invalidated individually by ArmUpdateTranslationTableEntry(). So drop the clean/invalidate of the caches, and only perform the full TLB flush if UpdateSectionEntries() was called, or if sections were split by UpdatePageEntries(). Finally, make the cache maintenance on the remapped regions themselves conditional on whether any memory type attributes were modified. Contributed-under: TianoCore Contribution Agreement 1.0 Signed-off-by: Ard Biesheuvel Reviewed-by: Leif Lindholm --- ArmPkg/Drivers/CpuDxe/Arm/Mmu.c | 60 +++++++++++--------- 1 file changed, 34 insertions(+), 26 deletions(-) -- 2.7.4 _______________________________________________ edk2-devel mailing list edk2-devel@lists.01.org https://lists.01.org/mailman/listinfo/edk2-devel diff --git a/ArmPkg/Drivers/CpuDxe/Arm/Mmu.c b/ArmPkg/Drivers/CpuDxe/Arm/Mmu.c index 16d6fcef9f1c..a2993cf16a35 100644 --- a/ArmPkg/Drivers/CpuDxe/Arm/Mmu.c +++ b/ArmPkg/Drivers/CpuDxe/Arm/Mmu.c @@ -347,10 +347,11 @@ SyncCacheConfig ( EFI_STATUS UpdatePageEntries ( - IN EFI_PHYSICAL_ADDRESS BaseAddress, - IN UINT64 Length, - IN UINT64 Attributes, - IN EFI_PHYSICAL_ADDRESS VirtualMask + IN EFI_PHYSICAL_ADDRESS BaseAddress, + IN UINT64 Length, + IN UINT64 Attributes, + IN EFI_PHYSICAL_ADDRESS VirtualMask, + OUT BOOLEAN *FlushTlbs OPTIONAL ) { EFI_STATUS Status; @@ -446,6 +447,9 @@ UpdatePageEntries ( // Re-read descriptor Descriptor = FirstLevelTable[FirstLevelIdx]; + if (FlushTlbs != NULL) { + *FlushTlbs = TRUE; + } } // Obtain page table base address @@ -471,15 +475,16 @@ UpdatePageEntries ( if (CurrentPageTableEntry != PageTableEntry) { Mva = (VOID *)(UINTN)((((UINTN)FirstLevelIdx) << TT_DESCRIPTOR_SECTION_BASE_SHIFT) + (PageTableIndex << TT_DESCRIPTOR_PAGE_BASE_SHIFT)); - if ((CurrentPageTableEntry & TT_DESCRIPTOR_PAGE_CACHEABLE_MASK) == TT_DESCRIPTOR_PAGE_CACHEABLE_MASK) { - // The current section mapping is cacheable so Clean/Invalidate the MVA of the page - // Note assumes switch(Attributes), not ARMv7 possibilities - WriteBackInvalidateDataCacheRange (Mva, TT_DESCRIPTOR_PAGE_SIZE); - } // Only need to update if we are changing the entry PageTable[PageTableIndex] = PageTableEntry; ArmUpdateTranslationTableEntry ((VOID *)&PageTable[PageTableIndex], Mva); + + // Clean/invalidate the cache for this page, but only + // if we are modifying the memory type attributes + if (((CurrentPageTableEntry ^ PageTableEntry) & TT_DESCRIPTOR_PAGE_CACHE_POLICY_MASK) != 0) { + WriteBackInvalidateDataCacheRange (Mva, TT_DESCRIPTOR_PAGE_SIZE); + } } Status = EFI_SUCCESS; @@ -581,7 +586,12 @@ UpdateSectionEntries ( // has this descriptor already been coverted to pages? if (TT_DESCRIPTOR_SECTION_TYPE_IS_PAGE_TABLE(CurrentDescriptor)) { // forward this 1MB range to page table function instead - Status = UpdatePageEntries ((FirstLevelIdx + i) << TT_DESCRIPTOR_SECTION_BASE_SHIFT, TT_DESCRIPTOR_SECTION_SIZE, Attributes, VirtualMask); + Status = UpdatePageEntries ( + (FirstLevelIdx + i) << TT_DESCRIPTOR_SECTION_BASE_SHIFT, + TT_DESCRIPTOR_SECTION_SIZE, + Attributes, + VirtualMask, + NULL); } else { // still a section entry @@ -596,15 +606,16 @@ UpdateSectionEntries ( if (CurrentDescriptor != Descriptor) { Mva = (VOID *)(UINTN)(((UINTN)FirstLevelTable) << TT_DESCRIPTOR_SECTION_BASE_SHIFT); - if ((CurrentDescriptor & TT_DESCRIPTOR_SECTION_CACHEABLE_MASK) == TT_DESCRIPTOR_SECTION_CACHEABLE_MASK) { - // The current section mapping is cacheable so Clean/Invalidate the MVA of the section - // Note assumes switch(Attributes), not ARMv7 possabilities - WriteBackInvalidateDataCacheRange (Mva, SIZE_1MB); - } // Only need to update if we are changing the descriptor FirstLevelTable[FirstLevelIdx + i] = Descriptor; ArmUpdateTranslationTableEntry ((VOID *)&FirstLevelTable[FirstLevelIdx + i], Mva); + + // Clean/invalidate the cache for this section, but only + // if we are modifying the memory type attributes + if (((CurrentDescriptor ^ Descriptor) & TT_DESCRIPTOR_SECTION_CACHE_POLICY_MASK) != 0) { + WriteBackInvalidateDataCacheRange (Mva, SIZE_1MB); + } } Status = EFI_SUCCESS; @@ -680,6 +691,7 @@ SetMemoryAttributes ( { EFI_STATUS Status; UINT64 ChunkLength; + BOOLEAN FlushTlbs; if (Length == 0) { return EFI_SUCCESS; @@ -692,6 +704,7 @@ SetMemoryAttributes ( return EFI_SUCCESS; } + FlushTlbs = FALSE; while (Length > 0) { if ((BaseAddress % TT_DESCRIPTOR_SECTION_SIZE == 0) && Length >= TT_DESCRIPTOR_SECTION_SIZE) { @@ -704,6 +717,8 @@ SetMemoryAttributes ( Status = UpdateSectionEntries (BaseAddress, ChunkLength, Attributes, VirtualMask); + + FlushTlbs = TRUE; } else { // @@ -721,7 +736,7 @@ SetMemoryAttributes ( BaseAddress, ChunkLength, Attributes)); Status = UpdatePageEntries (BaseAddress, ChunkLength, Attributes, - VirtualMask); + VirtualMask, &FlushTlbs); } if (EFI_ERROR (Status)) { @@ -732,16 +747,9 @@ SetMemoryAttributes ( Length -= ChunkLength; } - // Flush d-cache so descriptors make it back to uncached memory for subsequent table walks - // flush and invalidate pages - //TODO: Do we really need to invalidate the caches everytime we change the memory attributes ? - ArmCleanInvalidateDataCache (); - - ArmInvalidateInstructionCache (); - - // Invalidate all TLB entries so changes are synced - ArmInvalidateTlb (); - + if (FlushTlbs) { + ArmInvalidateTlb (); + } return Status; }