From patchwork Wed Mar 2 20:52:27 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Maydell X-Patchwork-Id: 547541 Delivered-To: patch@linaro.org Received: by 2002:ac0:f585:0:0:0:0:0 with SMTP id s5csp47293imq; Wed, 2 Mar 2022 13:24:24 -0800 (PST) X-Google-Smtp-Source: ABdhPJztu/HqdYhNy22HOUI1jT+3xN8gYEUelLjnKBakSoIbajIWv9zdw8y+un++x1EqxhoGja76 X-Received: by 2002:a81:48ca:0:b0:2dc:211e:b534 with SMTP id v193-20020a8148ca000000b002dc211eb534mr3307927ywa.84.1646256263967; Wed, 02 Mar 2022 13:24:23 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1646256263; cv=none; d=google.com; s=arc-20160816; b=PqujH9T6TSreXa9lg2O/026NOxDj3y2Adew9qArWjXnyQYDRQr3Jn+mUMHvVrM5x8t HwG5EjqptyZqQU/c0uEpl76+9Eo8C0iUUtzXnQn2N1XQwcXa5lMcABUhJ8cWJIyQ/pZz adEfDjueG+7Hzj4RIVzLBEJKAFkAbkeOBvVWa/7taagNSX5eIXXHWveP3woMvMN0EN5S r8OCFmAQyVTvUs/shMP7LQt5K+LHIyATqxvj3V5yQ7OFYBhmND2ek/nQwyOVJ+hun2uJ 96N8Ttr3hJPxj4h15XyBP3kWlRc+xmudC6f9mx6SfR8G0l1qUcvsXlR9VSsx6LgphvZd UDtg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=/7Vp2M9cs0sUqfL04/OQn8hXX+h842EPxz0wSYudyes=; b=I3sW9Gk4XOI0O0QHR5XAAIS7ThklYUcSSoOmZtCeltXynQq7hMb+E+XdHI5yz99pTq OMHuKXdIpb8RxvIqVP5iIBF5FnwZc9wuAs3ulYbdE4MRF0T/oKY55Ic+kJSv2omhrGk/ PvQsRuVP2Sd+woAFWV98+9g9w/fbZUxG7iJecKHoAV9HV3AApIxO/IlhLVOXBMdwgh8C Q0eCHmKTPKtDR9fR9IFutUFxqmB1NQ/abVzuUXzMYaQ2qKuRld1q3YTHYpHSvhJlekWi pRe6Nkq+fKFpQOJdk6XsLe0w57oBakY6vWOLKLaxZUPC78fSOtKEedLDLXIQTsufWJH3 lTqA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=K5RVJ7KR; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id be16-20020a05690c009000b002dbf9150095si109534ywb.5.2022.03.02.13.24.23 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Wed, 02 Mar 2022 13:24:23 -0800 (PST) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=K5RVJ7KR; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:50096 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1nPWSN-0005Br-Ck for patch@linaro.org; Wed, 02 Mar 2022 16:24:23 -0500 Received: from eggs.gnu.org ([209.51.188.92]:43654) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nPVxx-0005uz-R0 for qemu-devel@nongnu.org; Wed, 02 Mar 2022 15:52:57 -0500 Received: from [2a00:1450:4864:20::436] (port=37885 helo=mail-wr1-x436.google.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1nPVxv-0000CW-IX for qemu-devel@nongnu.org; Wed, 02 Mar 2022 15:52:57 -0500 Received: by mail-wr1-x436.google.com with SMTP id bk29so4690259wrb.4 for ; Wed, 02 Mar 2022 12:52:53 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=/7Vp2M9cs0sUqfL04/OQn8hXX+h842EPxz0wSYudyes=; b=K5RVJ7KRUEgcbYg0GLwiDe10/prviNiH/5lqscawDpESa4eo1ZfIzyVBT16auJWGVU XUuutO/Wjr312L8/hr8HXSjUvcvM5Q/mDiuOUHrc3w9jvsvI/31KDuPAB5WpqCY0EEgy s64XVsEYawY9yYy5Pz4+1vpbqZ6XXeRMVnkp0kx4nLu7bZowX+SI5cd5m9efb96F2Gh7 wNpfwvYGsQsowU9w/jtI7SypbuaeKOB1DuhZqB0EUIRKp5xtBXFOqJTrzENsv2sA57pd VrsJs3yZ1SimaCSkGjcIosdY0b65zlk8x9xHFqAYHVpAMCQnyvmzoAoa+I4DLtnBkIpM 948g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=/7Vp2M9cs0sUqfL04/OQn8hXX+h842EPxz0wSYudyes=; b=8OVTiecCbQFSMLpSwAi8C8fd7+SsMaJU4EMPuYZCthp9W7zf4BFVFQr8o9M58q9iYd Vpl32WZPmUdtQKA+sh8Zkkflyv1JG9mg7yNxdkZ3fj7NJlNXx4TI/Q954b7hCo+zWnzF iB50IlbioEjNwGVewr/foaOv9RlPO2YuBDygyWG15h0MPwO6AOZt9BdlLHNnuGyn2K+7 j/+SjgOQIF8COZ+q84+MQcHZ1nVXdi9tLuVe1xFkJEbCPXlmwPd3rmvuuEddOGn37uqj yXciYYSnRN+iisFmbuhEZdsNHc7Zkh/zOUYTdITlD+zlROua35j//Ky1EkhPu47nmCAe SQOg== X-Gm-Message-State: AOAM530R9sQigH+AnwUWd6AZCZHslk+TixS+PFeXpPaUhSBV9SAahyy+ irZG/CR3RcjCVqdVSqPC29KvqZuYK/r35Q== X-Received: by 2002:a05:6000:1882:b0:1f0:4c38:6d55 with SMTP id a2-20020a056000188200b001f04c386d55mr412663wri.483.1646254372845; Wed, 02 Mar 2022 12:52:52 -0800 (PST) Received: from orth.archaic.org.uk (orth.archaic.org.uk. [2001:8b0:1d0::2]) by smtp.gmail.com with ESMTPSA id x13-20020adfec0d000000b001e31279cc38sm90801wrn.11.2022.03.02.12.52.52 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 02 Mar 2022 12:52:52 -0800 (PST) From: Peter Maydell To: qemu-devel@nongnu.org Subject: [PULL 23/26] target/arm: Implement FEAT_LPA2 Date: Wed, 2 Mar 2022 20:52:27 +0000 Message-Id: <20220302205230.2122390-24-peter.maydell@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220302205230.2122390-1-peter.maydell@linaro.org> References: <20220302205230.2122390-1-peter.maydell@linaro.org> MIME-Version: 1.0 X-Host-Lookup-Failed: Reverse DNS lookup failed for 2a00:1450:4864:20::436 (failed) Received-SPF: pass client-ip=2a00:1450:4864:20::436; envelope-from=peter.maydell@linaro.org; helo=mail-wr1-x436.google.com X-Spam_score_int: -6 X-Spam_score: -0.7 X-Spam_bar: / X-Spam_report: (-0.7 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, PDS_HP_HELO_NORDNS=0.659, RCVD_IN_DNSWL_NONE=-0.0001, RDNS_NONE=0.793, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=no autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" From: Richard Henderson This feature widens physical addresses (and intermediate physical addresses for 2-stage translation) from 48 to 52 bits, when using 4k or 16k pages. This introduces the DS bit to TCR_ELx, which is RES0 unless the page size is enabled and supports LPA2, resulting in the effective value of DS for a given table walk. The DS bit changes the format of the page table descriptor slightly, moving the PS field out to TCR so that all pages have the same sharability and repurposing those bits of the page table descriptor for the highest bits of the output address. Do not yet enable FEAT_LPA2; we need extra plumbing to avoid tickling an old kernel bug. Reviewed-by: Peter Maydell Signed-off-by: Richard Henderson Message-id: 20220301215958.157011-17-richard.henderson@linaro.org Signed-off-by: Peter Maydell --- docs/system/arm/emulation.rst | 1 + target/arm/cpu.h | 22 ++++++++ target/arm/internals.h | 2 + target/arm/helper.c | 102 +++++++++++++++++++++++++++++----- 4 files changed, 112 insertions(+), 15 deletions(-) diff --git a/docs/system/arm/emulation.rst b/docs/system/arm/emulation.rst index 0053ddce208..520fd39071e 100644 --- a/docs/system/arm/emulation.rst +++ b/docs/system/arm/emulation.rst @@ -25,6 +25,7 @@ the following architecture extensions: - FEAT_JSCVT (JavaScript conversion instructions) - FEAT_LOR (Limited ordering regions) - FEAT_LPA (Large Physical Address space) +- FEAT_LPA2 (Large Physical and virtual Address space v2) - FEAT_LRCPC (Load-acquire RCpc instructions) - FEAT_LRCPC2 (Load-acquire RCpc instructions v2) - FEAT_LSE (Large System Extensions) diff --git a/target/arm/cpu.h b/target/arm/cpu.h index c52d56f6699..24d9fff1705 100644 --- a/target/arm/cpu.h +++ b/target/arm/cpu.h @@ -4284,6 +4284,28 @@ static inline bool isar_feature_aa64_i8mm(const ARMISARegisters *id) return FIELD_EX64(id->id_aa64isar1, ID_AA64ISAR1, I8MM) != 0; } +static inline bool isar_feature_aa64_tgran4_lpa2(const ARMISARegisters *id) +{ + return FIELD_SEX64(id->id_aa64mmfr0, ID_AA64MMFR0, TGRAN4) >= 1; +} + +static inline bool isar_feature_aa64_tgran4_2_lpa2(const ARMISARegisters *id) +{ + unsigned t = FIELD_EX64(id->id_aa64mmfr0, ID_AA64MMFR0, TGRAN4_2); + return t >= 3 || (t == 0 && isar_feature_aa64_tgran4_lpa2(id)); +} + +static inline bool isar_feature_aa64_tgran16_lpa2(const ARMISARegisters *id) +{ + return FIELD_EX64(id->id_aa64mmfr0, ID_AA64MMFR0, TGRAN16) >= 2; +} + +static inline bool isar_feature_aa64_tgran16_2_lpa2(const ARMISARegisters *id) +{ + unsigned t = FIELD_EX64(id->id_aa64mmfr0, ID_AA64MMFR0, TGRAN16_2); + return t >= 3 || (t == 0 && isar_feature_aa64_tgran16_lpa2(id)); +} + static inline bool isar_feature_aa64_ccidx(const ARMISARegisters *id) { return FIELD_EX64(id->id_aa64mmfr2, ID_AA64MMFR2, CCIDX) != 0; diff --git a/target/arm/internals.h b/target/arm/internals.h index 00af41d7925..a34be2e4595 100644 --- a/target/arm/internals.h +++ b/target/arm/internals.h @@ -1056,6 +1056,7 @@ static inline uint32_t aarch64_pstate_valid_mask(const ARMISARegisters *id) typedef struct ARMVAParameters { unsigned tsz : 8; unsigned ps : 3; + unsigned sh : 2; unsigned select : 1; bool tbi : 1; bool epd : 1; @@ -1063,6 +1064,7 @@ typedef struct ARMVAParameters { bool using16k : 1; bool using64k : 1; bool tsz_oob : 1; /* tsz has been clamped to legal range */ + bool ds : 1; } ARMVAParameters; ARMVAParameters aa64_va_parameters(CPUARMState *env, uint64_t va, diff --git a/target/arm/helper.c b/target/arm/helper.c index 3a7f5cf6f08..088956eecf0 100644 --- a/target/arm/helper.c +++ b/target/arm/helper.c @@ -4546,6 +4546,14 @@ static TLBIRange tlbi_aa64_get_range(CPUARMState *env, ARMMMUIdx mmuidx, } else { ret.base = extract64(value, 0, 37); } + if (param.ds) { + /* + * With DS=1, BaseADDR is always shifted 16 so that it is able + * to address all 52 va bits. The input address is perforce + * aligned on a 64k boundary regardless of translation granule. + */ + page_shift = 16; + } ret.base <<= page_shift; return ret; @@ -11081,8 +11089,13 @@ static bool check_s2_mmu_setup(ARMCPU *cpu, bool is_aa64, int level, const int grainsize = stride + 3; int startsizecheck; - /* Negative levels are never allowed. */ - if (level < 0) { + /* + * Negative levels are usually not allowed... + * Except for FEAT_LPA2, 4k page table, 52-bit address space, which + * begins with level -1. Note that previous feature tests will have + * eliminated this combination if it is not enabled. + */ + if (level < (inputsize == 52 && stride == 9 ? -1 : 0)) { return false; } @@ -11223,8 +11236,9 @@ ARMVAParameters aa64_va_parameters(CPUARMState *env, uint64_t va, ARMMMUIdx mmu_idx, bool data) { uint64_t tcr = regime_tcr(env, mmu_idx)->raw_tcr; - bool epd, hpd, using16k, using64k, tsz_oob; - int select, tsz, tbi, max_tsz, min_tsz, ps; + bool epd, hpd, using16k, using64k, tsz_oob, ds; + int select, tsz, tbi, max_tsz, min_tsz, ps, sh; + ARMCPU *cpu = env_archcpu(env); if (!regime_has_2_ranges(mmu_idx)) { select = 0; @@ -11238,7 +11252,9 @@ ARMVAParameters aa64_va_parameters(CPUARMState *env, uint64_t va, hpd = extract32(tcr, 24, 1); } epd = false; + sh = extract32(tcr, 12, 2); ps = extract32(tcr, 16, 3); + ds = extract64(tcr, 32, 1); } else { /* * Bit 55 is always between the two regions, and is canonical for @@ -11248,6 +11264,7 @@ ARMVAParameters aa64_va_parameters(CPUARMState *env, uint64_t va, if (!select) { tsz = extract32(tcr, 0, 6); epd = extract32(tcr, 7, 1); + sh = extract32(tcr, 12, 2); using64k = extract32(tcr, 14, 1); using16k = extract32(tcr, 15, 1); hpd = extract64(tcr, 41, 1); @@ -11257,24 +11274,51 @@ ARMVAParameters aa64_va_parameters(CPUARMState *env, uint64_t va, using64k = tg == 3; tsz = extract32(tcr, 16, 6); epd = extract32(tcr, 23, 1); + sh = extract32(tcr, 28, 2); hpd = extract64(tcr, 42, 1); } ps = extract64(tcr, 32, 3); + ds = extract64(tcr, 59, 1); } - if (cpu_isar_feature(aa64_st, env_archcpu(env))) { + if (cpu_isar_feature(aa64_st, cpu)) { max_tsz = 48 - using64k; } else { max_tsz = 39; } + /* + * DS is RES0 unless FEAT_LPA2 is supported for the given page size; + * adjust the effective value of DS, as documented. + */ min_tsz = 16; if (using64k) { - if (cpu_isar_feature(aa64_lva, env_archcpu(env))) { + if (cpu_isar_feature(aa64_lva, cpu)) { + min_tsz = 12; + } + ds = false; + } else if (ds) { + switch (mmu_idx) { + case ARMMMUIdx_Stage2: + case ARMMMUIdx_Stage2_S: + if (using16k) { + ds = cpu_isar_feature(aa64_tgran16_2_lpa2, cpu); + } else { + ds = cpu_isar_feature(aa64_tgran4_2_lpa2, cpu); + } + break; + default: + if (using16k) { + ds = cpu_isar_feature(aa64_tgran16_lpa2, cpu); + } else { + ds = cpu_isar_feature(aa64_tgran4_lpa2, cpu); + } + break; + } + if (ds) { min_tsz = 12; } } - /* TODO: FEAT_LPA2 */ if (tsz > max_tsz) { tsz = max_tsz; @@ -11296,6 +11340,7 @@ ARMVAParameters aa64_va_parameters(CPUARMState *env, uint64_t va, return (ARMVAParameters) { .tsz = tsz, .ps = ps, + .sh = sh, .select = select, .tbi = tbi, .epd = epd, @@ -11303,6 +11348,7 @@ ARMVAParameters aa64_va_parameters(CPUARMState *env, uint64_t va, .using16k = using16k, .using64k = using64k, .tsz_oob = tsz_oob, + .ds = ds, }; } @@ -11528,10 +11574,19 @@ static bool get_phys_addr_lpae(CPUARMState *env, uint64_t address, * VTCR_EL2.SL0 field (whose interpretation depends on the page size) */ uint32_t sl0 = extract32(tcr->raw_tcr, 6, 2); + uint32_t sl2 = extract64(tcr->raw_tcr, 33, 1); uint32_t startlevel; bool ok; - if (!aarch64 || stride == 9) { + /* SL2 is RES0 unless DS=1 & 4kb granule. */ + if (param.ds && stride == 9 && sl2) { + if (sl0 != 0) { + level = 0; + fault_type = ARMFault_Translation; + goto do_fault; + } + startlevel = -1; + } else if (!aarch64 || stride == 9) { /* AArch32 or 4KB pages */ startlevel = 2 - sl0; @@ -11585,10 +11640,12 @@ static bool get_phys_addr_lpae(CPUARMState *env, uint64_t address, * for both v7 and v8. However, for v8 the SBZ bits [47:40] must be 0 * or an AddressSize fault is raised. So for v8 we extract those SBZ * bits as part of the address, which will be checked via outputsize. - * For AArch64, the address field always goes up to bit 47 (with extra - * bits for FEAT_LPA placed elsewhere). AArch64 implies v8. + * For AArch64, the address field goes up to bit 47, or 49 with FEAT_LPA2; + * the highest bits of a 52-bit output are placed elsewhere. */ - if (arm_feature(env, ARM_FEATURE_V8)) { + if (param.ds) { + descaddrmask = MAKE_64BIT_MASK(0, 50); + } else if (arm_feature(env, ARM_FEATURE_V8)) { descaddrmask = MAKE_64BIT_MASK(0, 48); } else { descaddrmask = MAKE_64BIT_MASK(0, 40); @@ -11623,11 +11680,16 @@ static bool get_phys_addr_lpae(CPUARMState *env, uint64_t address, /* * For FEAT_LPA and PS=6, bits [51:48] of descaddr are in [15:12] - * of descriptor. Otherwise, if descaddr is out of range, raise - * AddressSizeFault. + * of descriptor. For FEAT_LPA2 and effective DS, bits [51:50] of + * descaddr are in [9:8]. Otherwise, if descaddr is out of range, + * raise AddressSizeFault. */ if (outputsize > 48) { - descaddr |= extract64(descriptor, 12, 4) << 48; + if (param.ds) { + descaddr |= extract64(descriptor, 8, 2) << 50; + } else { + descaddr |= extract64(descriptor, 12, 4) << 48; + } } else if (descaddr >> outputsize) { fault_type = ARMFault_AddressSize; goto do_fault; @@ -11721,7 +11783,17 @@ static bool get_phys_addr_lpae(CPUARMState *env, uint64_t address, assert(attrindx <= 7); cacheattrs->attrs = extract64(mair, attrindx * 8, 8); } - cacheattrs->shareability = extract32(attrs, 6, 2); + + /* + * For FEAT_LPA2 and effective DS, the SH field in the attributes + * was re-purposed for output address bits. The SH attribute in + * that case comes from TCR_ELx, which we extracted earlier. + */ + if (param.ds) { + cacheattrs->shareability = param.sh; + } else { + cacheattrs->shareability = extract32(attrs, 6, 2); + } *phys_ptr = descaddr; *page_size_ptr = page_size;