From patchwork Mon Mar 17 14:53:23 2014 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ian Campbell X-Patchwork-Id: 26385 Return-Path: X-Original-To: linaro@patches.linaro.org Delivered-To: linaro@patches.linaro.org Received: from mail-pb0-f69.google.com (mail-pb0-f69.google.com [209.85.160.69]) by ip-10-151-82-157.ec2.internal (Postfix) with ESMTPS id 946BB20143 for ; Mon, 17 Mar 2014 14:55:58 +0000 (UTC) Received: by mail-pb0-f69.google.com with SMTP id md12sf14508171pbc.8 for ; Mon, 17 Mar 2014 07:55:57 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:delivered-to:from:to:date:message-id:in-reply-to :references:mime-version:cc:subject:precedence:list-id :list-unsubscribe:list-post:list-help:list-subscribe:sender :errors-to:x-original-sender:x-original-authentication-results :mailing-list:list-archive:content-type:content-transfer-encoding; bh=XleqjCFgOAyDkIO0TR0fT0zpiMljhqf6/iPV3jCob2E=; b=GJkehqt7basVICA9Xg4Y2lZpGhksVLRLoyR63U/j+9dt+UVxV0T6oexYANGmBt6CR1 7wgS2QrQCiC8dXMC3/lYmr0MxtAm8xBmB+kucAuBI73c6WF7smyYZgKKg2n4EOYupD08 s4gxFsU64uGPG7Wz3OKIpkfxDx/b4l1ozmkwgIF2Xdv1YqlX/fUMtCYMqKjJGaBCkdRF O9rmYD6Z4z+Vp0JidfhjHZyJp+cuE28Yk7iSiqBsU0ydDLBpWoyjAv08bze090ls9hHD fV4r7AbjhyGDofz4AX46ga8TkXCusA9D8eK/ZmJsBZuwiL5yA1g8aT717F8aFm3OjSoH B0nQ== X-Gm-Message-State: ALoCoQmyhilK6p5vxUTT33F74JEjo/TzWPajadZoBHqdlsH6JlmGILjJt7es9nwbPl5uVjnsmpel X-Received: by 10.66.221.137 with SMTP id qe9mr9586223pac.4.1395068157819; Mon, 17 Mar 2014 07:55:57 -0700 (PDT) X-BeenThere: patchwork-forward@linaro.org Received: by 10.140.80.115 with SMTP id b106ls1697595qgd.97.gmail; Mon, 17 Mar 2014 07:55:57 -0700 (PDT) X-Received: by 10.58.201.5 with SMTP id jw5mr19485779vec.6.1395068157663; Mon, 17 Mar 2014 07:55:57 -0700 (PDT) Received: from mail-ve0-f175.google.com (mail-ve0-f175.google.com [209.85.128.175]) by mx.google.com with ESMTPS id sh5si1805064vdc.158.2014.03.17.07.55.57 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Mon, 17 Mar 2014 07:55:57 -0700 (PDT) Received-SPF: neutral (google.com: 209.85.128.175 is neither permitted nor denied by best guess record for domain of patch+caf_=patchwork-forward=linaro.org@linaro.org) client-ip=209.85.128.175; Received: by mail-ve0-f175.google.com with SMTP id oz11so5717822veb.6 for ; Mon, 17 Mar 2014 07:55:57 -0700 (PDT) X-Received: by 10.221.22.71 with SMTP id qv7mr452998vcb.34.1395068157532; Mon, 17 Mar 2014 07:55:57 -0700 (PDT) X-Forwarded-To: patchwork-forward@linaro.org X-Forwarded-For: patch@linaro.org patchwork-forward@linaro.org Delivered-To: patch@linaro.org Received: by 10.220.78.9 with SMTP id i9csp133135vck; Mon, 17 Mar 2014 07:55:57 -0700 (PDT) X-Received: by 10.220.81.194 with SMTP id y2mr933751vck.29.1395068157056; Mon, 17 Mar 2014 07:55:57 -0700 (PDT) Received: from lists.xen.org (lists.xen.org. [50.57.142.19]) by mx.google.com with ESMTPS id tz5si1808656vdc.151.2014.03.17.07.55.56 for (version=TLSv1 cipher=RC4-SHA bits=128/128); Mon, 17 Mar 2014 07:55:57 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of xen-devel-bounces@lists.xen.org designates 50.57.142.19 as permitted sender) client-ip=50.57.142.19; Received: from localhost ([127.0.0.1] helo=lists.xen.org) by lists.xen.org with esmtp (Exim 4.72) (envelope-from ) id 1WPYv2-0001y5-Jp; Mon, 17 Mar 2014 14:53:36 +0000 Received: from mail6.bemta4.messagelabs.com ([85.158.143.247]) by lists.xen.org with esmtp (Exim 4.72) (envelope-from ) id 1WPYv0-0001vq-5J for xen-devel@lists.xen.org; Mon, 17 Mar 2014 14:53:34 +0000 Received: from [85.158.143.35:34925] by server-3.bemta-4.messagelabs.com id 1C/B9-13602-D6C07235; Mon, 17 Mar 2014 14:53:33 +0000 X-Env-Sender: Ian.Campbell@citrix.com X-Msg-Ref: server-9.tower-21.messagelabs.com!1395068011!2629860!1 X-Originating-IP: [66.165.176.89] X-SpamReason: No, hits=0.0 required=7.0 tests=sa_preprocessor: VHJ1c3RlZCBJUDogNjYuMTY1LjE3Ni44OSA9PiAyMDMwMDc=\n X-StarScan-Received: X-StarScan-Version: 6.11.1; banners=-,-,- X-VirusChecked: Checked Received: (qmail 21991 invoked from network); 17 Mar 2014 14:53:32 -0000 Received: from smtp.citrix.com (HELO SMTP.CITRIX.COM) (66.165.176.89) by server-9.tower-21.messagelabs.com with RC4-SHA encrypted SMTP; 17 Mar 2014 14:53:32 -0000 X-IronPort-AV: E=Sophos;i="4.97,670,1389744000"; d="scan'208";a="112135714" Received: from accessns.citrite.net (HELO FTLPEX01CL02.citrite.net) ([10.9.154.239]) by FTLPIPO01.CITRIX.COM with ESMTP; 17 Mar 2014 14:53:31 +0000 Received: from norwich.cam.xci-test.com (10.80.248.129) by smtprelay.citrix.com (10.13.107.79) with Microsoft SMTP Server id 14.2.342.4; Mon, 17 Mar 2014 10:53:30 -0400 Received: from drall.uk.xensource.com ([10.80.16.71] helo=drall.uk.xensource.com.) by norwich.cam.xci-test.com with esmtp (Exim 4.72) (envelope-from ) id 1WPYuw-00012w-Cx; Mon, 17 Mar 2014 14:53:30 +0000 From: Ian Campbell To: Date: Mon, 17 Mar 2014 14:53:23 +0000 Message-ID: <1395068010-23344-1-git-send-email-ian.campbell@citrix.com> X-Mailer: git-send-email 1.7.10.4 In-Reply-To: <1395067981.18221.35.camel@kazak.uk.xensource.com> References: <1395067981.18221.35.camel@kazak.uk.xensource.com> MIME-Version: 1.0 X-DLP: MIA2 Cc: julien.grall@linaro.org, tim@xen.org, Ian Campbell , stefano.stabellini@eu.citrix.com Subject: [Xen-devel] [PATCH=v3 1/8] xen: arm: map memory as inner shareable. X-BeenThere: xen-devel@lists.xen.org X-Mailman-Version: 2.1.13 Precedence: list List-Id: List-Unsubscribe: , List-Post: , List-Help: , List-Subscribe: , Sender: xen-devel-bounces@lists.xen.org Errors-To: xen-devel-bounces@lists.xen.org X-Removed-Original-Auth: Dkim didn't pass. X-Original-Sender: ian.campbell@citrix.com X-Original-Authentication-Results: mx.google.com; spf=neutral (google.com: 209.85.128.175 is neither permitted nor denied by best guess record for domain of patch+caf_=patchwork-forward=linaro.org@linaro.org) smtp.mail=patch+caf_=patchwork-forward=linaro.org@linaro.org Mailing-list: list patchwork-forward@linaro.org; contact patchwork-forward+owners@linaro.org X-Google-Group-Id: 836684582541 List-Archive: The inner shareable domain contains all SMP processors, including different clusters (e.g. big.LITTLE). Therefore this is the correct thing to use for Xen memory mappings. The outer shareable domain is for devices on busses which are coherent and barrier-aware (e.g. AMBA4 AXI with ACE). While the system domain is for things behind bridges which are not. One wrinkle is that Normal memory with attributes Inner Non-cacheable, Outer Non-cacheable (which we call BUFFERABLE) must be mapped Outer Shareable on ARM v7. Therefore change the prototype of mfn_to_xen_entry to take the attribute index so we can DTRT. On ARMv8 the sharability is ignored and considered to always be Outer Shareable. Don't adjust the barriers, flushes etc, those remain as they were (which is more than is now required). I'll change those in a later patch. Many thanks to Leif for explaining the difference between Inner- and Outer-Shareable in words of two or less syllables, I hope I've replicated that explanation properly above! Signed-off-by: Ian Campbell Acked-by: Julien Grall --- v2: split dsb sy changes into a separate patch comment clarifications from Leif. mfn_to_p2m_entry sets shareability based on mattr, dev mappings remain outer. --- xen/arch/arm/arm32/head.S | 8 ++++---- xen/arch/arm/arm64/head.S | 8 ++++---- xen/arch/arm/mm.c | 34 +++++++++++++++++++--------------- xen/arch/arm/p2m.c | 18 ++++++++++++++++-- xen/include/asm-arm/page.h | 37 ++++++++++++++++++++++++++++++++++--- 5 files changed, 77 insertions(+), 28 deletions(-) diff --git a/xen/arch/arm/arm32/head.S b/xen/arch/arm/arm32/head.S index 72cda34..39978cb 100644 --- a/xen/arch/arm/arm32/head.S +++ b/xen/arch/arm/arm32/head.S @@ -24,8 +24,8 @@ #define ZIMAGE_MAGIC_NUMBER 0x016f2818 -#define PT_PT 0xe7f /* nG=1 AF=1 SH=10 AP=01 NS=1 ATTR=111 T=1 P=1 */ -#define PT_MEM 0xe7d /* nG=1 AF=1 SH=10 AP=01 NS=1 ATTR=111 T=0 P=1 */ +#define PT_PT 0xf7f /* nG=1 AF=1 SH=11 AP=01 NS=1 ATTR=111 T=1 P=1 */ +#define PT_MEM 0xf7d /* nG=1 AF=1 SH=11 AP=01 NS=1 ATTR=111 T=0 P=1 */ #define PT_DEV 0xe71 /* nG=1 AF=1 SH=10 AP=01 NS=1 ATTR=100 T=0 P=1 */ #define PT_DEV_L3 0xe73 /* nG=1 AF=1 SH=10 AP=01 NS=1 ATTR=100 T=1 P=1 */ @@ -224,10 +224,10 @@ cpu_init_done: mcr CP32(r1, HMAIR1) /* Set up the HTCR: - * PT walks use Outer-Shareable accesses, + * PT walks use Inner-Shareable accesses, * PT walks are write-back, write-allocate in both cache levels, * Full 32-bit address space goes through this table. */ - ldr r0, =0x80002500 + ldr r0, =0x80003500 mcr CP32(r0, HTCR) /* Set up the HSCTLR: diff --git a/xen/arch/arm/arm64/head.S b/xen/arch/arm/arm64/head.S index d151724..9547ef5 100644 --- a/xen/arch/arm/arm64/head.S +++ b/xen/arch/arm/arm64/head.S @@ -25,8 +25,8 @@ #include #include -#define PT_PT 0xe7f /* nG=1 AF=1 SH=10 AP=01 NS=1 ATTR=111 T=1 P=1 */ -#define PT_MEM 0xe7d /* nG=1 AF=1 SH=10 AP=01 NS=1 ATTR=111 T=0 P=1 */ +#define PT_PT 0xf7f /* nG=1 AF=1 SH=11 AP=01 NS=1 ATTR=111 T=1 P=1 */ +#define PT_MEM 0xf7d /* nG=1 AF=1 SH=11 AP=01 NS=1 ATTR=111 T=0 P=1 */ #define PT_DEV 0xe71 /* nG=1 AF=1 SH=10 AP=01 NS=1 ATTR=100 T=0 P=1 */ #define PT_DEV_L3 0xe73 /* nG=1 AF=1 SH=10 AP=01 NS=1 ATTR=100 T=1 P=1 */ @@ -227,10 +227,10 @@ skip_bss: /* Set up the HTCR: * PASize -- 40 bits / 1TB * Top byte is used - * PT walks use Outer-Shareable accesses, + * PT walks use Inner-Shareable accesses, * PT walks are write-back, write-allocate in both cache levels, * Full 64-bit address space goes through this table. */ - ldr x0, =0x80822500 + ldr x0, =0x80823500 msr tcr_el2, x0 /* Set up the SCTLR_EL2: diff --git a/xen/arch/arm/mm.c b/xen/arch/arm/mm.c index 305879f..b7360b6 100644 --- a/xen/arch/arm/mm.c +++ b/xen/arch/arm/mm.c @@ -212,9 +212,8 @@ void dump_hyp_walk(vaddr_t addr) /* Map a 4k page in a fixmap entry */ void set_fixmap(unsigned map, unsigned long mfn, unsigned attributes) { - lpae_t pte = mfn_to_xen_entry(mfn); + lpae_t pte = mfn_to_xen_entry(mfn, attributes); pte.pt.table = 1; /* 4k mappings always have this bit set */ - pte.pt.ai = attributes; pte.pt.xn = 1; write_pte(xen_fixmap + third_table_offset(FIXMAP_ADDR(map)), pte); flush_xen_data_tlb_range_va(FIXMAP_ADDR(map), PAGE_SIZE); @@ -270,7 +269,7 @@ void *map_domain_page(unsigned long mfn) else if ( map[slot].pt.avail == 0 ) { /* Commandeer this 2MB slot */ - pte = mfn_to_xen_entry(slot_mfn); + pte = mfn_to_xen_entry(slot_mfn, WRITEALLOC); pte.pt.avail = 1; write_pte(map + slot, pte); break; @@ -397,7 +396,7 @@ static inline lpae_t pte_of_xenaddr(vaddr_t va) { paddr_t ma = va + phys_offset; unsigned long mfn = ma >> PAGE_SHIFT; - return mfn_to_xen_entry(mfn); + return mfn_to_xen_entry(mfn, WRITEALLOC); } void __init remove_early_mappings(void) @@ -418,6 +417,12 @@ void __init setup_pagetables(unsigned long boot_phys_offset, paddr_t xen_paddr) lpae_t pte, *p; int i; + /* Map the destination in the boot misc area. */ + dest_va = BOOT_RELOC_VIRT_START; + pte = mfn_to_xen_entry(xen_paddr >> PAGE_SHIFT, WRITEALLOC); + write_pte(xen_second + second_table_offset(dest_va), pte); + flush_xen_data_tlb_range_va(dest_va, SECOND_SIZE); + /* Calculate virt-to-phys offset for the new location */ phys_offset = xen_paddr - (unsigned long) _start; @@ -451,7 +456,7 @@ void __init setup_pagetables(unsigned long boot_phys_offset, paddr_t xen_paddr) /* Initialise xen second level entries ... */ /* ... Xen's text etc */ - pte = mfn_to_xen_entry(xen_paddr>>PAGE_SHIFT); + pte = mfn_to_xen_entry(xen_paddr>>PAGE_SHIFT, WRITEALLOC); pte.pt.xn = 0;/* Contains our text mapping! */ xen_second[second_table_offset(XEN_VIRT_START)] = pte; @@ -466,7 +471,7 @@ void __init setup_pagetables(unsigned long boot_phys_offset, paddr_t xen_paddr) /* Map the destination in the boot misc area. */ dest_va = BOOT_RELOC_VIRT_START; - pte = mfn_to_xen_entry(xen_paddr >> PAGE_SHIFT); + pte = mfn_to_xen_entry(xen_paddr >> PAGE_SHIFT, WRITEALLOC); write_pte(boot_second + second_table_offset(dest_va), pte); flush_xen_data_tlb_range_va(dest_va, SECOND_SIZE); #ifdef CONFIG_ARM_64 @@ -495,7 +500,7 @@ void __init setup_pagetables(unsigned long boot_phys_offset, paddr_t xen_paddr) unsigned long va = XEN_VIRT_START + (i << PAGE_SHIFT); if ( !is_kernel(va) ) break; - pte = mfn_to_xen_entry(mfn); + pte = mfn_to_xen_entry(mfn, WRITEALLOC); pte.pt.table = 1; /* 4k mappings always have this bit set */ if ( is_kernel_text(va) || is_kernel_inittext(va) ) { @@ -565,7 +570,7 @@ int init_secondary_pagetables(int cpu) * domheap mapping pages. */ for ( i = 0; i < DOMHEAP_SECOND_PAGES; i++ ) { - pte = mfn_to_xen_entry(virt_to_mfn(domheap+i*LPAE_ENTRIES)); + pte = mfn_to_xen_entry(virt_to_mfn(domheap+i*LPAE_ENTRIES), WRITEALLOC); pte.pt.table = 1; write_pte(&first[first_table_offset(DOMHEAP_VIRT_START+i*FIRST_SIZE)], pte); } @@ -610,7 +615,7 @@ static void __init create_32mb_mappings(lpae_t *second, count = nr_mfns / LPAE_ENTRIES; p = second + second_linear_offset(virt_offset); - pte = mfn_to_xen_entry(base_mfn); + pte = mfn_to_xen_entry(base_mfn, WRITEALLOC); pte.pt.contig = 1; /* These maps are in 16-entry contiguous chunks. */ for ( i = 0; i < count; i++ ) { @@ -682,13 +687,13 @@ void __init setup_xenheap_mappings(unsigned long base_mfn, else { unsigned long first_mfn = alloc_boot_pages(1, 1); - pte = mfn_to_xen_entry(first_mfn); + pte = mfn_to_xen_entry(first_mfn, WRITEALLOC); pte.pt.table = 1; write_pte(p, pte); first = mfn_to_virt(first_mfn); } - pte = mfn_to_xen_entry(base_mfn); + pte = mfn_to_xen_entry(base_mfn, WRITEALLOC); /* TODO: Set pte.pt.contig when appropriate. */ write_pte(&first[first_table_offset(vaddr)], pte); @@ -724,7 +729,7 @@ void __init setup_frametable_mappings(paddr_t ps, paddr_t pe) second = mfn_to_virt(second_base); for ( i = 0; i < nr_second; i++ ) { - pte = mfn_to_xen_entry(second_base + i); + pte = mfn_to_xen_entry(second_base + i, WRITEALLOC); pte.pt.table = 1; write_pte(&xen_first[first_table_offset(FRAMETABLE_VIRT_START)+i], pte); } @@ -776,7 +781,7 @@ static int create_xen_table(lpae_t *entry) if ( p == NULL ) return -ENOMEM; clear_page(p); - pte = mfn_to_xen_entry(virt_to_mfn(p)); + pte = mfn_to_xen_entry(virt_to_mfn(p), WRITEALLOC); pte.pt.table = 1; write_pte(entry, pte); return 0; @@ -822,9 +827,8 @@ static int create_xen_entries(enum xenmap_operation op, addr, mfn); return -EINVAL; } - pte = mfn_to_xen_entry(mfn); + pte = mfn_to_xen_entry(mfn, ai); pte.pt.table = 1; - pte.pt.ai = ai; write_pte(&third[third_table_offset(addr)], pte); break; case REMOVE: diff --git a/xen/arch/arm/p2m.c b/xen/arch/arm/p2m.c index d00c882..b9d8ca6 100644 --- a/xen/arch/arm/p2m.c +++ b/xen/arch/arm/p2m.c @@ -145,10 +145,10 @@ static lpae_t mfn_to_p2m_entry(unsigned long mfn, unsigned int mattr, p2m_type_t t) { paddr_t pa = ((paddr_t) mfn) << PAGE_SHIFT; - /* xn and write bit will be defined in the switch */ + /* sh, xn and write bit will be defined in the following switches + * based on mattr and t. */ lpae_t e = (lpae_t) { .p2m.af = 1, - .p2m.sh = LPAE_SH_OUTER, .p2m.read = 1, .p2m.mattr = mattr, .p2m.table = 1, @@ -158,6 +158,20 @@ static lpae_t mfn_to_p2m_entry(unsigned long mfn, unsigned int mattr, BUILD_BUG_ON(p2m_max_real_type > (1 << 4)); + switch (mattr) + { + case MATTR_MEM: + e.p2m.sh = LPAE_SH_INNER; + break; + + case MATTR_DEV: + e.p2m.sh = LPAE_SH_OUTER; + break; + default: + BUG(); + break; + } + switch (t) { case p2m_ram_rw: diff --git a/xen/include/asm-arm/page.h b/xen/include/asm-arm/page.h index 905beb8..fd22993 100644 --- a/xen/include/asm-arm/page.h +++ b/xen/include/asm-arm/page.h @@ -185,7 +185,7 @@ typedef union { /* Standard entry type that we'll use to build Xen's own pagetables. * We put the same permissions at every level, because they're ignored * by the walker in non-leaf entries. */ -static inline lpae_t mfn_to_xen_entry(unsigned long mfn) +static inline lpae_t mfn_to_xen_entry(unsigned long mfn, unsigned attr) { paddr_t pa = ((paddr_t) mfn) << PAGE_SHIFT; lpae_t e = (lpae_t) { @@ -193,10 +193,9 @@ static inline lpae_t mfn_to_xen_entry(unsigned long mfn) .xn = 1, /* No need to execute outside .text */ .ng = 1, /* Makes TLB flushes easier */ .af = 1, /* No need for access tracking */ - .sh = LPAE_SH_OUTER, /* Xen mappings are globally coherent */ .ns = 1, /* Hyp mode is in the non-secure world */ .user = 1, /* See below */ - .ai = WRITEALLOC, + .ai = attr, .table = 0, /* Set to 1 for links and 4k maps */ .valid = 1, /* Mappings are present */ }};; @@ -205,6 +204,38 @@ static inline lpae_t mfn_to_xen_entry(unsigned long mfn) * pagetables un User mode it's OK. If this changes, remember * to update the hard-coded values in head.S too */ + switch ( attr ) + { + case BUFFERABLE: + /* + * ARM ARM: Overlaying the shareability attribute (DDI + * 0406C.b B3-1376 to 1377) + * + * A memory region with a resultant memory type attribute of Normal, + * and a resultant cacheability attribute of Inner Non-cacheable, + * Outer Non-cacheable, must have a resultant shareability attribute + * of Outer Shareable, otherwise shareability is UNPREDICTABLE. + * + * On ARMv8 sharability is ignored and explicitly treated as Outer + * Shareable for Normal Inner Non_cacheable, Outer Non-cacheable. + */ + e.pt.sh = LPAE_SH_OUTER; + break; + case UNCACHED: + case DEV_SHARED: + /* Shareability is ignored for non-Normal memory, Outer is as + * good as anything. + * + * On ARMv8 sharability is ignored and explicitly treated as Outer + * Shareable for any device memory type. + */ + e.pt.sh = LPAE_SH_OUTER; + break; + default: + e.pt.sh = LPAE_SH_INNER; /* Xen mappings are SMP coherent */ + break; + } + ASSERT(!(pa & ~PAGE_MASK)); ASSERT(!(pa & ~PADDR_MASK));