From patchwork Wed Jun 3 01:12:36 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 187403 Delivered-To: patch@linaro.org Received: by 2002:a92:cf06:0:0:0:0:0 with SMTP id c6csp69745ilo; Tue, 2 Jun 2020 18:13:51 -0700 (PDT) X-Google-Smtp-Source: ABdhPJzCBQ96rUzDUNGO8dKExa69f4bjBFDeGo9nGhQ8c5vh2CBxs289S+EUMAwkhEMt7n3fyQ4p X-Received: by 2002:a5b:10:: with SMTP id a16mr37916609ybp.262.1591146831548; Tue, 02 Jun 2020 18:13:51 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1591146831; cv=none; d=google.com; s=arc-20160816; b=WXP3ImNt5St3D1LxQPORvdvnzZNEZuM0hO9Mj5YpjjeG2+z8Ngy6wpqE1lo1LdZBgY ITapbKwCfFdYTLbaPxCGFiCvwh9lj2k6t159p93JMlK2fWFBL/UKX4hDK2NTZZQheojZ u8dBbXSNR4S641r+J1AEqX4ZcSaNEPMSJiJE5BXckonswX76OoVYtu5m7UThZ1mnrXJc BeqekwTXaQtQsCthgpzR0n1AgUe/Vl4RbcCqOtQI0LrNCFpdE17jB0pKwWhpr9CUTKhJ ThtyMnrEcg/DrK3r738CPI08J9GRydZnwWnnGaaZBcS5v14nt2UKYeGScPKkA4lGfOW9 HOEQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=OzobXxmD62KwDEFdjrZjAAighBDtrTCRu38jKjN27BM=; b=N1oPKZIQ90Z4zhuMDVKBycRDGVAimLrVgFosOPo9htlq7hbo/+jsQJtqoRTvvzLoHT somqoaFGu+HHUax3fZzvLCmoPVKgK7mC47R71GY4aDQonwNsY9dIK4umETNcgw+8ozNO LLJi7T0JoMxcA1zPuAVRqUxn+FNjdGJD4RxrYFD5ofC4dAUoEFTpVBodFfvvvWJrbjA/ VkriDophWXjSzgg3sWfIZg5YupbnDMuzncOOfVEoQlMqX8XPRRhmL0WsGpnO8fArUWPu 0Aaz2IVNV9HNJcFcpfwo/Vb4zaP1ofdjWBZRs2llAWifPEbdW/scFTZjNIK+75qcxxfk 4vWA== ARC-Authentication-Results: i=1; mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=ZT0kiUc1; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id z10si282334ybk.485.2020.06.02.18.13.51 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 02 Jun 2020 18:13:51 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=ZT0kiUc1; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:45644 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jgHyY-0003d4-Ty for patch@linaro.org; Tue, 02 Jun 2020 21:13:50 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:48496) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jgHy6-0003HL-Oq for qemu-devel@nongnu.org; Tue, 02 Jun 2020 21:13:22 -0400 Received: from mail-pg1-x52e.google.com ([2607:f8b0:4864:20::52e]:33855) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1jgHy5-0003U9-MN for qemu-devel@nongnu.org; Tue, 02 Jun 2020 21:13:22 -0400 Received: by mail-pg1-x52e.google.com with SMTP id m1so586101pgk.1 for ; Tue, 02 Jun 2020 18:13:21 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=OzobXxmD62KwDEFdjrZjAAighBDtrTCRu38jKjN27BM=; b=ZT0kiUc1Ujlo87PPHi9f6LpaacySTV6JqK01cnZQbuxrdlllwF9L9JfwR+IPMtpQo9 eXepl4920HFBg9PcADbYfdDMUiWrkAwB7miHPLicJPYoAfuA5uS+IBaNwiIz2GEfyY1N 4eRMidlP2GNbGI9eliSbGzrbmGcDK7vInBFbC+LRLj6jICk16fXlncNUg+S6sNQ5/9pv xNRQTzxw5/TyCq7H9UFqX0eN+Ytnwn/pF/qHy/jc0iQ/HkwY2g32D/17tWj11D1Mo/CI YJrw1KFUk4VPlKKYqqi2PAIdVi7RY4DsjNQYdB9lSH4/EuHR8td81wRKHGHoA/adcsX8 AWwA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=OzobXxmD62KwDEFdjrZjAAighBDtrTCRu38jKjN27BM=; b=o4rhejfAeFClj0kSvyfi5RKNWS8cc1oOscy1XNLH0qunjlEgJLqLyhFTf8BVZnxu9l 5GJcWxWO4wC8or9IT2hAmjLkjr7LhefgL0jMcgtx9Lxam0oR/w/nAlpG5/x+s6DXucu7 uIIDuYGPq8y31EwDhAkQI/jQqHYG7AdSO96SjSdLjNmo+hhEOm7XoNYWGJ+SSD8KRWb1 zgLv4tH/mRacJ2BzA5O/TQ2/ZsrXChfk5UijCj8E0wN26Wm+1ruJRIJ7QC+OynOlkzEj yev22gSmrRoiDpDHKvaBTatNtoqxHmMCBSCP5CS1sGUsRjFiw/ehggL3uI0rM3gV1xYw Zx+w== X-Gm-Message-State: AOAM5323P+XBqa9CwkAPMfyZPNQhheCEUby4qWHKA1Sve7qi/i0urs5S MBVRRj/gpnRdX/HcvklGH3PeTa9ePGU= X-Received: by 2002:aa7:8b56:: with SMTP id i22mr20028587pfd.63.1591146800024; Tue, 02 Jun 2020 18:13:20 -0700 (PDT) Received: from localhost.localdomain (174-21-143-238.tukw.qwest.net. [174.21.143.238]) by smtp.gmail.com with ESMTPSA id 3sm290067pfe.85.2020.06.02.18.13.18 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 02 Jun 2020 18:13:19 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v7 01/42] target/arm: Add isar tests for mte Date: Tue, 2 Jun 2020 18:12:36 -0700 Message-Id: <20200603011317.473934-2-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200603011317.473934-1-richard.henderson@linaro.org> References: <20200603011317.473934-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::52e; envelope-from=richard.henderson@linaro.org; helo=mail-pg1-x52e.google.com X-detected-operating-system: by eggs.gnu.org: No matching host in p0f cache. That's all we know. X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: peter.maydell@linaro.org, qemu-arm@nongnu.org, steplong@quicinc.com Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" Signed-off-by: Richard Henderson --- target/arm/cpu.h | 10 ++++++++++ 1 file changed, 10 insertions(+) -- 2.25.1 Reviewed-by: Peter Maydell diff --git a/target/arm/cpu.h b/target/arm/cpu.h index 677584e5da..f8ac11e73b 100644 --- a/target/arm/cpu.h +++ b/target/arm/cpu.h @@ -3814,6 +3814,16 @@ static inline bool isar_feature_aa64_bti(const ARMISARegisters *id) return FIELD_EX64(id->id_aa64pfr1, ID_AA64PFR1, BT) != 0; } +static inline bool isar_feature_aa64_mte_insn_reg(const ARMISARegisters *id) +{ + return FIELD_EX64(id->id_aa64pfr1, ID_AA64PFR1, MTE) != 0; +} + +static inline bool isar_feature_aa64_mte(const ARMISARegisters *id) +{ + return FIELD_EX64(id->id_aa64pfr1, ID_AA64PFR1, MTE) >= 2; +} + static inline bool isar_feature_aa64_pmu_8_1(const ARMISARegisters *id) { return FIELD_EX64(id->id_aa64dfr0, ID_AA64DFR0, PMUVER) >= 4 && From patchwork Wed Jun 3 01:12:37 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 187404 Delivered-To: patch@linaro.org Received: by 2002:a92:cf06:0:0:0:0:0 with SMTP id c6csp69777ilo; Tue, 2 Jun 2020 18:13:54 -0700 (PDT) X-Google-Smtp-Source: ABdhPJyfoGjxUgHsFq95TMQ8fhj1Ppx0C7AK8D2n6ky3BO2vyOa8d+6kc3XMrCRj7d0sFXVgFVOE X-Received: by 2002:a25:9d0e:: with SMTP id i14mr45227218ybp.64.1591146834420; Tue, 02 Jun 2020 18:13:54 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1591146834; cv=none; d=google.com; s=arc-20160816; b=iXXLLcjc8d+oOz77z/2xlvRBSGIbZEQfLXss8xTJhTOPa2AkG7wj+Rvz+aFw01NjNe PbxZp3BMwCd/xIIVA+8L47Vaty8Z4LXodqFyV1xRBATWwHsr8LcfXeiqwXfV7oLhk63r h3bmPVm0jEFV2OrKC2lFzCZgjIM+V7tGqKNZXX16wNkggXJL3PLUdZKAQCONHKTNW+xy uJvYKLCjwKsgj+rTFBys91bn2AUxP96EXWiWRYRemL5Nq3zNeUww8rLvHtdLr1TjNxhm INJZSOR4BmkVKRVeeK22vygGqbznAbU4okpvoeyRM/hIVABuTo3DM+ATxzrqa5d9q/WM SCzw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=LadquP7zcJjQQG0Yfbt1ahthtHYVsuxaqjQXIxyHC+g=; b=Yuck5mhviiI6jj1qW0lyKZy3qsqBE2EJvdTvymr/Zxcfb0yJdTTbZBTa5Qx7OCGbOG pcEHffLpF6RzvGNUGggsvisI7YYR/811mv5drYy3yJuTtGF+x7MuSAjkj6CDsaBf8wO5 2rwKjlPcWsskNCdto/zH8IQtunDEgIGj/kpcs2lerTNgtmTIjCjkwbAMQqILk+JTi9RA WsT/W/tBilvXKqFaDdB0kUu8Zp6WpfHPkodRM3onUVoj52UK4rR8umz5+57uchvg3kt4 zxMGMSf4AH+X+2JpLxLtBaZWcnqYQo0fodDPF3QgKIJr4PFGmyErAu9Lx8ddHt5q/mcJ em1w== ARC-Authentication-Results: i=1; mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b="u+UtE/59"; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id y7si327097yba.215.2020.06.02.18.13.54 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 02 Jun 2020 18:13:54 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b="u+UtE/59"; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:45990 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jgHyb-0003mD-Rb for patch@linaro.org; Tue, 02 Jun 2020 21:13:53 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:48508) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jgHy7-0003Hg-EF for qemu-devel@nongnu.org; Tue, 02 Jun 2020 21:13:23 -0400 Received: from mail-pf1-x444.google.com ([2607:f8b0:4864:20::444]:44130) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1jgHy6-0003Vb-RJ for qemu-devel@nongnu.org; Tue, 02 Jun 2020 21:13:23 -0400 Received: by mail-pf1-x444.google.com with SMTP id f3so400817pfd.11 for ; Tue, 02 Jun 2020 18:13:22 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=LadquP7zcJjQQG0Yfbt1ahthtHYVsuxaqjQXIxyHC+g=; b=u+UtE/59Le3pHvHBcznGutnq/Jy4yqyhDnjhwwE2lSJCdgfZbgodRX9j6tPmJ8hYKu IdnRKB4rB+wkqH4mUHceRFWxAwA+OTsvl5tx9+IxRUKNtL/4HY+/h+gai/CrpCZUKGFx sXxt/WII8p055eMQ/Dm1e5pf5eP6XrWQ4CDJ1VxgJeuheWUqgN5SRZacYXE5ch5p1Foj CY0nbaC9gUtZq6E8FIlCYYLpocgWCSsuaEqkJICoVySJnHBjSDAzuGqyN0yd4LJGlfI2 v6Z6hlFEDsmlcA/6XIIPTglrdWZc0V3ii2qMN1ZSq8V2NknLycAiPVMV89694bIa2l/O u4TQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=LadquP7zcJjQQG0Yfbt1ahthtHYVsuxaqjQXIxyHC+g=; b=UtAgpIZwRsH/Myv7jW5ITZ9SFs1QHP2K64lknYLGJvDbGQw2K2NE31k8FRdkcK3HR2 PQRV5Kx1yfwLCokmC8tDBw2Dam8+BkO8NELuafRKa749sE4iuI+Umq7duYAqUhi5o2Vf lTl33WAfgY1xpxfXGF/+K4t2ZX7vxdr5SGBF+zZdIIfe2kIqUgdiMKvLKspcyWd5R/eL ap8Ju9uMTh/pmsb4S13B8OYrmpAuWJRQ1vEPEGwb9v226VFL3xN43v42rctYrVhpMSIc rpaxo0m/wS/FQej96jzCU5+IU0/ctTi9mGXkASuYMWMOE3/ay28p8Xxhc9LB4x8g7MFz NNVw== X-Gm-Message-State: AOAM533uokG329KsTWsvYr0Fgj4iTLjU5Cb0f+cBKyuRRfV0nApa7gde 2lQZmA5MGUAHrhnPmeZ5fKAqQ1+oFKo= X-Received: by 2002:a17:90a:df16:: with SMTP id gp22mr2327443pjb.6.1591146801223; Tue, 02 Jun 2020 18:13:21 -0700 (PDT) Received: from localhost.localdomain (174-21-143-238.tukw.qwest.net. [174.21.143.238]) by smtp.gmail.com with ESMTPSA id 3sm290067pfe.85.2020.06.02.18.13.20 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 02 Jun 2020 18:13:20 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v7 02/42] target/arm: Improve masking of SCR RES0 bits Date: Tue, 2 Jun 2020 18:12:37 -0700 Message-Id: <20200603011317.473934-3-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200603011317.473934-1-richard.henderson@linaro.org> References: <20200603011317.473934-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::444; envelope-from=richard.henderson@linaro.org; helo=mail-pf1-x444.google.com X-detected-operating-system: by eggs.gnu.org: No matching host in p0f cache. That's all we know. X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: peter.maydell@linaro.org, qemu-arm@nongnu.org, steplong@quicinc.com Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" Protect reads of aa64 id registers with ARM_CP_STATE_AA64. Use this as a simpler test than arm_el_is_aa64, since EL3 cannot change mode. Signed-off-by: Richard Henderson --- target/arm/helper.c | 15 ++++++++------- 1 file changed, 8 insertions(+), 7 deletions(-) -- 2.25.1 Reviewed-by: Peter Maydell diff --git a/target/arm/helper.c b/target/arm/helper.c index a92ae55672..2ec49c1a55 100644 --- a/target/arm/helper.c +++ b/target/arm/helper.c @@ -2011,9 +2011,16 @@ static void scr_write(CPUARMState *env, const ARMCPRegInfo *ri, uint64_t value) uint32_t valid_mask = 0x3fff; ARMCPU *cpu = env_archcpu(env); - if (arm_el_is_aa64(env, 3)) { + if (ri->state == ARM_CP_STATE_AA64) { value |= SCR_FW | SCR_AW; /* these two bits are RES1. */ valid_mask &= ~SCR_NET; + + if (cpu_isar_feature(aa64_lor, cpu)) { + valid_mask |= SCR_TLOR; + } + if (cpu_isar_feature(aa64_pauth, cpu)) { + valid_mask |= SCR_API | SCR_APK; + } } else { valid_mask &= ~(SCR_RW | SCR_ST); } @@ -2032,12 +2039,6 @@ static void scr_write(CPUARMState *env, const ARMCPRegInfo *ri, uint64_t value) valid_mask &= ~SCR_SMD; } } - if (cpu_isar_feature(aa64_lor, cpu)) { - valid_mask |= SCR_TLOR; - } - if (cpu_isar_feature(aa64_pauth, cpu)) { - valid_mask |= SCR_API | SCR_APK; - } /* Clear all-context RES0 bits. */ value &= valid_mask; From patchwork Wed Jun 3 01:12:38 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 187401 Delivered-To: patch@linaro.org Received: by 2002:a92:cf06:0:0:0:0:0 with SMTP id c6csp69648ilo; Tue, 2 Jun 2020 18:13:41 -0700 (PDT) X-Google-Smtp-Source: ABdhPJx5ewPSMSItAgig3jflUAWV70oaH2d6WSGKBUE9GhEzOxsrbYdPzqE2DSaoYBRLVCh2Hksa X-Received: by 2002:a25:b442:: with SMTP id c2mr49543350ybg.273.1591146821023; Tue, 02 Jun 2020 18:13:41 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1591146821; cv=none; d=google.com; s=arc-20160816; b=HBRosunXIb/iKYvBXCfyD0Wv+varaXFm7w6SEW9owhlBTTSxYLpqfuWJKbW1f0qdoH B3XuekPYa53KkIqKH8A5LwUPSngOY3fXHc/K34MY/PXjTuL0nYojFKXvH0HV5GCawsp7 Z8gwyCxdNjc0qcipmhTYRLQqzX65d8a23+ePOaHDE89JMQAdo1C8yRg/Smg3rwlhgk6L ge4qJ4Y6ROJ/AoixhWsAeLw8JHiXBZt5dUSY8Q3YHZqqOqTzwmy7yolzya1eYTZr2b6g saUPvQOnOrrL0cKWT4keHXjsOO1OAK3ctUReMiIAh4PVkSIJJrOfAP8OfaiwuMKPSYlG VtHA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=w0iAAPJiPUMN5WMvx91TGcplu0Afl0eDKA3EhLW5gNo=; b=lXCCOhbGWdCCSsCNw9mHJIIAssj/d+1j4bYrbIeUKQl43fcNa3T4AYnstp8CEke1Qq K6R70CN3fne6I5m4f0C2qpL+8leu2a4Sq/Y203VIQ0j6koz/wBbA+BLH+EmPrgmwlirB RhKVmQzU/1JODhWMWRgexHcKvxoVkBzVAtMqXaaASsfZpVGzFIwHcoZc8/6YG0dqcTyj 3cjGfycLRrU38L/6GdrR51hbeeB0wGSh2fz1RASSesu2vXdQiZGxN+/2IPlEfGyzv4Aw QKLGLYkfpKWczINe98fWwBKd3xdEhs8DgCgHI8Pymdk7f/+5Ik4uzhSh10TTg53HJCri 8cdA== ARC-Authentication-Results: i=1; mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=PFmMIelk; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id d30si229776yba.165.2020.06.02.18.13.40 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 02 Jun 2020 18:13:41 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=PFmMIelk; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:44980 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jgHyN-0003Ki-Ge for patch@linaro.org; Tue, 02 Jun 2020 21:13:39 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:48522) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jgHy8-0003JJ-Ko for qemu-devel@nongnu.org; Tue, 02 Jun 2020 21:13:24 -0400 Received: from mail-pf1-x444.google.com ([2607:f8b0:4864:20::444]:37010) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1jgHy8-0003W6-0o for qemu-devel@nongnu.org; Tue, 02 Jun 2020 21:13:24 -0400 Received: by mail-pf1-x444.google.com with SMTP id j1so258215pfe.4 for ; Tue, 02 Jun 2020 18:13:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=w0iAAPJiPUMN5WMvx91TGcplu0Afl0eDKA3EhLW5gNo=; b=PFmMIelkaLMfRJ69igouLY4t2p0h6QeXvGx/htpKH8meFoeZa6CHwBtSs8fnerkGZi 1Ip6W5CFanpbjzkUSueDx7t5yRm9nYJ4X/ekZZZBskew3VrMo/5tS1/C+hYtH7NmBZw0 pLK04b01a4DazOrnUcyaZVwnwXmzf7omD5n+/CPR9DEenRrHGCNo1RG6sTRP39s4R1Pj sXiDeJHCGeFS0PVSZ7HRY4wotTlvApfdsvQwOD+B3twvbdVLydn0CVfBl9dKUq0BUlou vKc72Lhjg100ZorDtU1rqVT8CYAgjg5Lwwri4fh/z3zQ9f1ph/yXG25dRtQlsP9Jrg9c foTA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=w0iAAPJiPUMN5WMvx91TGcplu0Afl0eDKA3EhLW5gNo=; b=C0QS7wbbWiG8MB9GYQCp9lyfmdhj8ulNDWt5iTFNsxb6JqU7Z/IJDbU82fLWf9yKWg FEyM3kLSSjwmRipzQSAjeMLdHQZqR38uzKwRMKgpE6Geo+JqTscgSfl+DlZHsIyWQWn3 02P4K/gZ0Y2KtvlfGVyb8Z/ZeiVHJQFja+6byxmm9JAqBX64jIgfQj30pGtoxmhK0mBO ECv58HFMo0rkYyRDskM4omfctLOxBLOy2hF/GtwkcWBNvXdO75vorL1cQHmhAKpHrx7c 5urMjtVPIKgsL8WPk2x9LO8v2D4jSp6U7uZ3AkrslsPPNokVE0fPvHSB4cBnH5qkI+YP k0pg== X-Gm-Message-State: AOAM530xxnQxGeXY1k3AWvC2+DshbTZ4iLbrrsIFB7IfzHcxRyj3cRfp HjSqjg+wTU5UD6Twkvrxd1gqJkpP378= X-Received: by 2002:a17:90a:5806:: with SMTP id h6mr2442263pji.66.1591146802500; Tue, 02 Jun 2020 18:13:22 -0700 (PDT) Received: from localhost.localdomain (174-21-143-238.tukw.qwest.net. [174.21.143.238]) by smtp.gmail.com with ESMTPSA id 3sm290067pfe.85.2020.06.02.18.13.21 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 02 Jun 2020 18:13:21 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v7 03/42] target/arm: Add support for MTE to SCTLR_ELx Date: Tue, 2 Jun 2020 18:12:38 -0700 Message-Id: <20200603011317.473934-4-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200603011317.473934-1-richard.henderson@linaro.org> References: <20200603011317.473934-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::444; envelope-from=richard.henderson@linaro.org; helo=mail-pf1-x444.google.com X-detected-operating-system: by eggs.gnu.org: No matching host in p0f cache. That's all we know. X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: peter.maydell@linaro.org, qemu-arm@nongnu.org, steplong@quicinc.com Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" This does not attempt to rectify all of the res0 bits, but does clear the mte bits when not enabled. Since there is no high-part mapping of SCTLR, aa32 mode cannot write to these bits. Signed-off-by: Richard Henderson --- target/arm/helper.c | 23 +++++++++++++++++------ 1 file changed, 17 insertions(+), 6 deletions(-) -- 2.25.1 Reviewed-by: Peter Maydell diff --git a/target/arm/helper.c b/target/arm/helper.c index 2ec49c1a55..7862bf502d 100644 --- a/target/arm/helper.c +++ b/target/arm/helper.c @@ -4698,6 +4698,22 @@ static void sctlr_write(CPUARMState *env, const ARMCPRegInfo *ri, { ARMCPU *cpu = env_archcpu(env); + if (arm_feature(env, ARM_FEATURE_PMSA) && !cpu->has_mpu) { + /* M bit is RAZ/WI for PMSA with no MPU implemented */ + value &= ~SCTLR_M; + } + + /* ??? Lots of these bits are not implemented. */ + + if (ri->state == ARM_CP_STATE_AA64 && !cpu_isar_feature(aa64_mte, cpu)) { + if (ri->opc1 == 6) { /* SCTLR_EL3 */ + value &= ~(SCTLR_ITFSB | SCTLR_TCF | SCTLR_ATA); + } else { + value &= ~(SCTLR_ITFSB | SCTLR_TCF0 | SCTLR_TCF | + SCTLR_ATA0 | SCTLR_ATA); + } + } + if (raw_read(env, ri) == value) { /* Skip the TLB flush if nothing actually changed; Linux likes * to do a lot of pointless SCTLR writes. @@ -4705,13 +4721,8 @@ static void sctlr_write(CPUARMState *env, const ARMCPRegInfo *ri, return; } - if (arm_feature(env, ARM_FEATURE_PMSA) && !cpu->has_mpu) { - /* M bit is RAZ/WI for PMSA with no MPU implemented */ - value &= ~SCTLR_M; - } - raw_write(env, ri, value); - /* ??? Lots of these bits are not implemented. */ + /* This may enable/disable the MMU, so do a TLB flush. */ tlb_flush(CPU(cpu)); From patchwork Wed Jun 3 01:12:39 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 187407 Delivered-To: patch@linaro.org Received: by 2002:a92:cf06:0:0:0:0:0 with SMTP id c6csp71200ilo; Tue, 2 Jun 2020 18:16:17 -0700 (PDT) X-Google-Smtp-Source: ABdhPJx7bztEQaAz7coQohpAJuV+PFI1QaK1BBh9hLOu4pyJN8AdyBCk2yDhLXJnwcChTHq+WmNN X-Received: by 2002:a25:69c3:: with SMTP id e186mr22091141ybc.445.1591146977255; Tue, 02 Jun 2020 18:16:17 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1591146977; cv=none; d=google.com; s=arc-20160816; b=Ogm7fo6Ut3L55h+pGEc788vJiOdHF8vBPQ+xdHDGy3Yuw+gfP7+wPLBeNGQ2+qLO58 p+UBfhh381h6IwjKRMgv8v0vedN3njPkjDtRbE/7T8fBBM/jvyt0BplIV6fXmD7e9dLY VSxPJkuaaVCM1N9kwFHVmPZ4ca7WWKyTRGri0AQJyYiSCgKw9Xe4epZFPp4YJRyWLzgF Wn5dy1NmE23HTxj5Ff20E3cL8LvZlTkbHP6PVwC4+bNuOGZWce1Hcu7iWCWag1S6jF54 wvVBJ7+wcBETJaI/lVxLAvySHQXXf6c94X3k2K2L9+yJUbPFdCDjtxi6aba5rKF97c8A Nzqg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=ULSeXKjALQhZKETBeRDnYwgqeM3FCy75bsyu+BRJV8g=; b=XUFpuEn9Vvu8UsViN01qmUMNJ8epMDS5EncrUrYJG28yMGaPssVGieIZoqSs/rTkDg OZBb4R6ldAU3ggZKYHd3FPOAvPY1iEa8utFgwa+m6itqmPsX+Zr2wBih85F8H1x9cVxS yOMFGz+xhaoHi2/Ha57kEE5uGBOX2dIk4h9FEtTfeKzb2XTFmiPDZBWglB6ZpI+SmUnA ab0+JPXKZrfUoaP8MglNUdjGny2dRgNOSC2FK/79jBh2S3utobZnku2EK9Nn1XUJgDS7 e9gp3oBHjl+hsSlYwWXNFMDyii157fzK7zzYSV2JocspLn+2c7nv5bXjf41kXNQonc8F HYXQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=L87U40S6; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id n85si326350ybg.397.2020.06.02.18.16.17 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 02 Jun 2020 18:16:17 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=L87U40S6; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:54516 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jgI0u-0007aX-Nb for patch@linaro.org; Tue, 02 Jun 2020 21:16:16 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:48536) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jgHy9-0003Lq-Ta for qemu-devel@nongnu.org; Tue, 02 Jun 2020 21:13:25 -0400 Received: from mail-pl1-x642.google.com ([2607:f8b0:4864:20::642]:41032) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1jgHy9-0003WL-8I for qemu-devel@nongnu.org; Tue, 02 Jun 2020 21:13:25 -0400 Received: by mail-pl1-x642.google.com with SMTP id y17so193179plb.8 for ; Tue, 02 Jun 2020 18:13:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=ULSeXKjALQhZKETBeRDnYwgqeM3FCy75bsyu+BRJV8g=; b=L87U40S6DgDwLa4TQPLjCJMjehB3mpworJDCJbTwwqjBUFLMjbUyrT1SnPT5dPYnZO glUwErvnWjSn9RM94qL+KI2/zw7PCgtK3rf4g+HmsrmNa1z8uTUN5EvqBEYnkTJ3ljDr kBTRYFFsY/uKl3Gi/H5u+C/RO6sjAaoXwN/cWsTd+5EaE0B7FoCn9rPeYcQW6MmVBxJ+ y5Xg2nPeAANL1vxAgCE5l4KWg/gIgkQi59BBfV8fO/c3PbxNHxEmT/9AmpKwJKW8qiqv k46SGHxBH2tPz29siOEL35jQFBVQdL6nz1Kda0PGFgvUuoA/dLW0dq++y+DqrAYh6hD2 b67A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=ULSeXKjALQhZKETBeRDnYwgqeM3FCy75bsyu+BRJV8g=; b=hiiaCFt2y6ga4HcanEEJ2olNMfYW2W662KIab3QQPebZsF2geZhwJmd+5Oj8mFTLdQ Wdr24EtbXi4c9nefJbNfKirlcwpO/DpdMxXeZ9BxVbo7S7D9/rwPJe1mfnnGqU49GAk+ 3ktaH1oy6CjMrmeNrvRIjCAsJNtqW3HS3sLcrZis8O7QsGB3v1xSh52Lj31Pmr8Ka3Qv jmj7C94rq6yCOHgiF6P8lJMBCufPwprFwNAsuzmYAoikAFdRal7a341iK0pT31ojAQhU wNsTm1je8kbSJcvDmwQzXwFRdZjGL/POyRFuIZtQk2Hg6Fxt5nil7VNhYtUh60xlfHIJ 149Q== X-Gm-Message-State: AOAM530Q/DOSWmxtu3CtNIUz0ACmCcPfLwlCR+x6RDVSGKOh3NZEB5a1 kBp9xiyAk/rWsuFVTv6WfGHI3hoPxrs= X-Received: by 2002:a17:902:eb03:: with SMTP id l3mr25830622plb.49.1591146803556; Tue, 02 Jun 2020 18:13:23 -0700 (PDT) Received: from localhost.localdomain (174-21-143-238.tukw.qwest.net. [174.21.143.238]) by smtp.gmail.com with ESMTPSA id 3sm290067pfe.85.2020.06.02.18.13.22 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 02 Jun 2020 18:13:22 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v7 04/42] target/arm: Add support for MTE to HCR_EL2 and SCR_EL3 Date: Tue, 2 Jun 2020 18:12:39 -0700 Message-Id: <20200603011317.473934-5-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200603011317.473934-1-richard.henderson@linaro.org> References: <20200603011317.473934-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::642; envelope-from=richard.henderson@linaro.org; helo=mail-pl1-x642.google.com X-detected-operating-system: by eggs.gnu.org: No matching host in p0f cache. That's all we know. X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: peter.maydell@linaro.org, qemu-arm@nongnu.org, steplong@quicinc.com Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" Signed-off-by: Richard Henderson --- target/arm/helper.c | 6 ++++++ 1 file changed, 6 insertions(+) -- 2.25.1 Reviewed-by: Peter Maydell diff --git a/target/arm/helper.c b/target/arm/helper.c index 7862bf502d..f2ead07ead 100644 --- a/target/arm/helper.c +++ b/target/arm/helper.c @@ -2021,6 +2021,9 @@ static void scr_write(CPUARMState *env, const ARMCPRegInfo *ri, uint64_t value) if (cpu_isar_feature(aa64_pauth, cpu)) { valid_mask |= SCR_API | SCR_APK; } + if (cpu_isar_feature(aa64_mte, cpu)) { + valid_mask |= SCR_ATA; + } } else { valid_mask &= ~(SCR_RW | SCR_ST); } @@ -5248,6 +5251,9 @@ static void do_hcr_write(CPUARMState *env, uint64_t value, uint64_t valid_mask) if (cpu_isar_feature(aa64_pauth, cpu)) { valid_mask |= HCR_API | HCR_APK; } + if (cpu_isar_feature(aa64_mte, cpu)) { + valid_mask |= HCR_ATA | HCR_TID5; + } } /* Clear RES0 bits. */ From patchwork Wed Jun 3 01:12:40 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 187405 Delivered-To: patch@linaro.org Received: by 2002:a92:cf06:0:0:0:0:0 with SMTP id c6csp70836ilo; Tue, 2 Jun 2020 18:15:41 -0700 (PDT) X-Google-Smtp-Source: ABdhPJyPuIJB05Jn7sLgrFPwjbdXu/CpfUePVICe01TJGRCp8+vuWgk7mNb4mxy1F7uR+BJptH3o X-Received: by 2002:a25:2602:: with SMTP id m2mr941180ybm.82.1591146941175; Tue, 02 Jun 2020 18:15:41 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1591146941; cv=none; d=google.com; s=arc-20160816; b=NqmcxY+GUCJPVEm4W1svIkQDbjL2Q+1nt4CoLLG9+CYgogUJkOeoyFGPWd/6sJ5AOY dGRoI7jO9Lm9PMi/r5mk79FQZGhauzTYudvOUJ0Vd9WfuHebSvgw7pE8CL7fHd762nOu OIVqUIx4FCVUjC3qea539X1Wl7VRofIeD1AI8GE7zVUummcf2+HvmFX7krf5UYqdgUiY qmTy7hCbuXjM4kGVH84T4Ma+yG0NS7SQlJwBBwVmA3WYGIv2Uxn2ZPr6B/LdHwp9pUCL 4V7xSSOeT0Gtmk3EvCkoXoIYn99rlANv3ddp3m4XOd+0O+Q3RJ5JYvzGA+WHL2ozXoMr DolQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=LFvCo0Qacf2rnL7CzQb0VBfQYmhjPV5RLkkRAA9+6zQ=; b=O36ljaZIEPZD9APaWDMqXJXUNZT93YX+U0zkSP83PAOaZRowSx+8boaBK8daO5mrsz edwfJ4tLVu/t5pkMouZ3gM/suy0oYTDS9lYjS6Y/Vt2OlV/7LtPRDjrkoE9zDOLYMiXU sfZkcQNXnvyesCq8QOMAMH88VKRx62w0WzTiXN/h3rvQihURqupszACheq3rmQkOe+WR 9BLep2g02vk6sDB3c62nveGt8oEemR2M8pAeIr07y9EWcOq2ZSx/U+ghx9LEUT4sJewN zmZ2ForDQ36SUsAPBGwSdKfDDceJCde6d8hpr+jZv3ZL5QVi8pI32utG2gsiJdH8+rc3 6zZQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=BszsxdEw; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id r9si305389ybm.154.2020.06.02.18.15.41 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 02 Jun 2020 18:15:41 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=BszsxdEw; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:54880 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jgI0J-0007lP-Kb for patch@linaro.org; Tue, 02 Jun 2020 21:15:40 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:48574) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jgHyD-0003V8-0u for qemu-devel@nongnu.org; Tue, 02 Jun 2020 21:13:29 -0400 Received: from mail-pf1-x433.google.com ([2607:f8b0:4864:20::433]:41242) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1jgHyA-0003Wl-S4 for qemu-devel@nongnu.org; Tue, 02 Jun 2020 21:13:28 -0400 Received: by mail-pf1-x433.google.com with SMTP id 64so410132pfg.8 for ; Tue, 02 Jun 2020 18:13:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=LFvCo0Qacf2rnL7CzQb0VBfQYmhjPV5RLkkRAA9+6zQ=; b=BszsxdEwd5+sXi/rk3DwYO9vi41GSZNh2dI1VFjI+eMKAWPojNWHyGX46OMtETz0g2 QZjpQN/OOukn5OSdSY7dWCRA7V39zZRs5PKfsrIHRhxm+e9U0UdzjoNrbgOOjJ9KRwkB qekIjaf3hxsyBvXuaaArfffhFtR+L7XvkRlxAONhXGFgGHV5clCGZS25kVitWT1OYArc 4ANZIVWJq01OdLVsnz6G+LbgCjPplL5CjzVpPt9vCgnFPfCVYtA0qUORmKWoXiQJVzMJ 13LDwwwHqwz93qR9QwNaoJrtkcW1oRCjgf8yeLjHQQdm1GNtxYBa325EQEk5SICxFN69 S0Hg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=LFvCo0Qacf2rnL7CzQb0VBfQYmhjPV5RLkkRAA9+6zQ=; b=X7KNIPwMZ8VOZVp2wfwg4z1gnT6dGLOmf27aReNKU7KgXj1FodUJUDgGfq9rcUQVvN O4MbVzsTCkV2q/ApjSx3ufxsAJkpsoaxaiJFJax+UrtpMjH84cxO0j5ccm6VfsPQc8ue mAU1QqRoa/03k/wFPS6d+S1dR6AZ/LDKITWoFfw97DtiOm9VJ4+Hj0aoF7fQ/C2RQP1J ng01b4OnyTyqpSvcNqzfKIZKcyXfY9M4qqPOa8CzKlFcILIOgBgvkx1Vglu2TaU8NL8T cVRlDDQd9/4/2OjSEipCWSbeSGqpE/n8/PspNPK1PEiz3hwuqpIuy6z7F/0lpnyLyhP8 qvNw== X-Gm-Message-State: AOAM531WnMEdB0M3Non3JLFUCNgwzJv9j/XNUCb3dylxXS9NkcTDsm4n JyOJaitLrqAb2TcWZK3n+6ahSgVC1Hc= X-Received: by 2002:a17:90a:c215:: with SMTP id e21mr2405516pjt.224.1591146805021; Tue, 02 Jun 2020 18:13:25 -0700 (PDT) Received: from localhost.localdomain (174-21-143-238.tukw.qwest.net. [174.21.143.238]) by smtp.gmail.com with ESMTPSA id 3sm290067pfe.85.2020.06.02.18.13.23 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 02 Jun 2020 18:13:24 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v7 05/42] target/arm: Rename DISAS_UPDATE to DISAS_UPDATE_EXIT Date: Tue, 2 Jun 2020 18:12:40 -0700 Message-Id: <20200603011317.473934-6-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200603011317.473934-1-richard.henderson@linaro.org> References: <20200603011317.473934-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::433; envelope-from=richard.henderson@linaro.org; helo=mail-pf1-x433.google.com X-detected-operating-system: by eggs.gnu.org: No matching host in p0f cache. That's all we know. X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: peter.maydell@linaro.org, qemu-arm@nongnu.org, steplong@quicinc.com Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" Emphasize that the is_jmp option exits to the main loop. Signed-off-by: Richard Henderson --- target/arm/translate.h | 14 ++++++++------ target/arm/translate-a64.c | 8 ++++---- target/arm/translate-vfp.inc.c | 2 +- target/arm/translate.c | 12 ++++++------ 4 files changed, 19 insertions(+), 17 deletions(-) -- 2.25.1 Reviewed-by: Peter Maydell diff --git a/target/arm/translate.h b/target/arm/translate.h index c937dfe9bf..68a55251c3 100644 --- a/target/arm/translate.h +++ b/target/arm/translate.h @@ -148,7 +148,8 @@ static inline void disas_set_insn_syndrome(DisasContext *s, uint32_t syn) /* is_jmp field values */ #define DISAS_JUMP DISAS_TARGET_0 /* only pc was modified dynamically */ -#define DISAS_UPDATE DISAS_TARGET_1 /* cpu state was modified dynamically */ +/* CPU state was modified dynamically; exit to main loop for interrupts. */ +#define DISAS_UPDATE_EXIT DISAS_TARGET_1 /* These instructions trap after executing, so the A32/T32 decoder must * defer them until after the conditional execution state has been updated. * WFI also needs special handling when single-stepping. @@ -164,11 +165,12 @@ static inline void disas_set_insn_syndrome(DisasContext *s, uint32_t syn) * custom end-of-TB code) */ #define DISAS_BX_EXCRET DISAS_TARGET_8 -/* For instructions which want an immediate exit to the main loop, - * as opposed to attempting to use lookup_and_goto_ptr. Unlike - * DISAS_UPDATE this doesn't write the PC on exiting the translation - * loop so you need to ensure something (gen_a64_set_pc_im or runtime - * helper) has done so before we reach return from cpu_tb_exec. +/* + * For instructions which want an immediate exit to the main loop, as opposed + * to attempting to use lookup_and_goto_ptr. Unlike DISAS_UPDATE_EXIT, this + * doesn't write the PC on exiting the translation loop so you need to ensure + * something (gen_a64_set_pc_im or runtime helper) has done so before we reach + * return from cpu_tb_exec. */ #define DISAS_EXIT DISAS_TARGET_9 diff --git a/target/arm/translate-a64.c b/target/arm/translate-a64.c index 874f3eb4f9..430f1867ab 100644 --- a/target/arm/translate-a64.c +++ b/target/arm/translate-a64.c @@ -1607,7 +1607,7 @@ static void handle_msr_i(DisasContext *s, uint32_t insn, gen_helper_msr_i_daifclear(cpu_env, t1); tcg_temp_free_i32(t1); /* For DAIFClear, exit the cpu loop to re-evaluate pending IRQs. */ - s->base.is_jmp = DISAS_UPDATE; + s->base.is_jmp = DISAS_UPDATE_EXIT; break; default: @@ -1786,7 +1786,7 @@ static void handle_sys(DisasContext *s, uint32_t insn, bool isread, if ((tb_cflags(s->base.tb) & CF_USE_ICOUNT) && (ri->type & ARM_CP_IO)) { /* I/O operations must end the TB here (whether read or write) */ - s->base.is_jmp = DISAS_UPDATE; + s->base.is_jmp = DISAS_UPDATE_EXIT; } if (!isread && !(ri->type & ARM_CP_SUPPRESS_TB_END)) { /* @@ -1801,7 +1801,7 @@ static void handle_sys(DisasContext *s, uint32_t insn, bool isread, * but allow this to be suppressed by the register definition * (usually only necessary to work around guest bugs). */ - s->base.is_jmp = DISAS_UPDATE; + s->base.is_jmp = DISAS_UPDATE_EXIT; } } @@ -14334,7 +14334,7 @@ static void aarch64_tr_tb_stop(DisasContextBase *dcbase, CPUState *cpu) gen_goto_tb(dc, 1, dc->base.pc_next); break; default: - case DISAS_UPDATE: + case DISAS_UPDATE_EXIT: gen_a64_set_pc_im(dc->base.pc_next); /* fall through */ case DISAS_EXIT: diff --git a/target/arm/translate-vfp.inc.c b/target/arm/translate-vfp.inc.c index e1a9017598..4e1fbe0dc0 100644 --- a/target/arm/translate-vfp.inc.c +++ b/target/arm/translate-vfp.inc.c @@ -2861,6 +2861,6 @@ static bool trans_VLLDM_VLSTM(DisasContext *s, arg_VLLDM_VLSTM *a) tcg_temp_free_i32(fptr); /* End the TB, because we have updated FP control bits */ - s->base.is_jmp = DISAS_UPDATE; + s->base.is_jmp = DISAS_UPDATE_EXIT; return true; } diff --git a/target/arm/translate.c b/target/arm/translate.c index c8296116d4..00a6fe6752 100644 --- a/target/arm/translate.c +++ b/target/arm/translate.c @@ -2931,7 +2931,7 @@ static void gen_msr_banked(DisasContext *s, int r, int sysm, int rn) tcg_temp_free_i32(tcg_tgtmode); tcg_temp_free_i32(tcg_regno); tcg_temp_free_i32(tcg_reg); - s->base.is_jmp = DISAS_UPDATE; + s->base.is_jmp = DISAS_UPDATE_EXIT; } static void gen_mrs_banked(DisasContext *s, int r, int sysm, int rn) @@ -2953,7 +2953,7 @@ static void gen_mrs_banked(DisasContext *s, int r, int sysm, int rn) tcg_temp_free_i32(tcg_tgtmode); tcg_temp_free_i32(tcg_regno); store_reg(s, rn, tcg_reg); - s->base.is_jmp = DISAS_UPDATE; + s->base.is_jmp = DISAS_UPDATE_EXIT; } /* Store value to PC as for an exception return (ie don't @@ -7312,7 +7312,7 @@ static void gen_srs(DisasContext *s, tcg_temp_free_i32(tmp); } tcg_temp_free_i32(addr); - s->base.is_jmp = DISAS_UPDATE; + s->base.is_jmp = DISAS_UPDATE_EXIT; } /* Generate a label used for skipping this instruction */ @@ -10361,7 +10361,7 @@ static bool trans_SETEND(DisasContext *s, arg_SETEND *a) } if (a->E != (s->be_data == MO_BE)) { gen_helper_setend(cpu_env); - s->base.is_jmp = DISAS_UPDATE; + s->base.is_jmp = DISAS_UPDATE_EXIT; } return true; } @@ -11084,7 +11084,7 @@ static void arm_tr_tb_stop(DisasContextBase *dcbase, CPUState *cpu) break; case DISAS_NEXT: case DISAS_TOO_MANY: - case DISAS_UPDATE: + case DISAS_UPDATE_EXIT: gen_set_pc_im(dc, dc->base.pc_next); /* fall through */ default: @@ -11111,7 +11111,7 @@ static void arm_tr_tb_stop(DisasContextBase *dcbase, CPUState *cpu) case DISAS_JUMP: gen_goto_ptr(); break; - case DISAS_UPDATE: + case DISAS_UPDATE_EXIT: gen_set_pc_im(dc, dc->base.pc_next); /* fall through */ default: From patchwork Wed Jun 3 01:12:41 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 187406 Delivered-To: patch@linaro.org Received: by 2002:a92:cf06:0:0:0:0:0 with SMTP id c6csp70865ilo; Tue, 2 Jun 2020 18:15:43 -0700 (PDT) X-Google-Smtp-Source: ABdhPJxzeyJHVa4dSAH9x9Y8iN3K1/Oo2wGi7oCvJ7TG4FD0/eoyTRJGTCKOKNP0v/AcBG2uYlbg X-Received: by 2002:a25:c054:: with SMTP id c81mr31221047ybf.76.1591146943843; Tue, 02 Jun 2020 18:15:43 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1591146943; cv=none; d=google.com; s=arc-20160816; b=VOStIS6QzUDl+NNAHrS4JC78jAF5kIO4n8o6O3Ckw9+F8yL7Sg1cCGOaMYOeOoCmol Ofc4fFXbriaih0yIMuehOiHDrM3mEeEKJIdDbOWxXjMrYpmcDu/YOf3CAp/8RHOK7TXU Ny9NreM/GItWf29k0KmruVljWnQeVCl3O+HapHLXUKJJ/vNq/QGok9hCMkOg21tx59qu rSBCC/A8K+tR0T099LRbNEdaCZfv6Cox2JvgXOTp7ZHd1a3HbCN/7slJ4rg/d1EhU6Ji Ic/gHpJcpQtaiMiT3Sgut2Om21P7Pg1lc7oDxkZK8j4MlJIkvVyycCCKjmeyRXfgyDeQ wYpw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=2dMQqpeyi0XKIpauyWT1oQVejUj1nmYGROEX25a4OR4=; b=yuIY1RRmIlTOwfqou5WUMcU8Ad3Bmsm6KAPjMDsx2CUKA8NfGIIigUgOZRQcBgg9Oj 1VkfHBIWd98AgLwpGg3bmJ6oc8M4b+IIIARmD1dl9bGcEInrOqR1pEQiXKK/89oPoJsJ i4ps91ivsGyJT2Ud3ccYNzZGG7Mm+Cnwi4ZxlsXYUjJk6bJv4v4h3oZAFk+fXjQSzZLK x/+Wd+mIosp1lT28tpK8SIxaAcZe4pdPZXIw/f64KOhx4kAEap6RhPjeV2FDWfV7pOur zouIpzZPYS3ocbbMuylKR7Y9Vnthjjio1GfSNSrIw+0xbtXRLu1ngdNxAuveOv9PWIKn FJRg== ARC-Authentication-Results: i=1; mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=P7yUf7QQ; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id d31si295844yba.474.2020.06.02.18.15.43 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 02 Jun 2020 18:15:43 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=P7yUf7QQ; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:55022 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jgI0N-0007ok-5V for patch@linaro.org; Tue, 02 Jun 2020 21:15:43 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:48572) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jgHyC-0003UP-PH for qemu-devel@nongnu.org; Tue, 02 Jun 2020 21:13:28 -0400 Received: from mail-pl1-x643.google.com ([2607:f8b0:4864:20::643]:36969) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1jgHyC-0003X4-1W for qemu-devel@nongnu.org; Tue, 02 Jun 2020 21:13:28 -0400 Received: by mail-pl1-x643.google.com with SMTP id y18so201724plr.4 for ; Tue, 02 Jun 2020 18:13:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=2dMQqpeyi0XKIpauyWT1oQVejUj1nmYGROEX25a4OR4=; b=P7yUf7QQBE5+/vpm46q5Q4PrJEeyCCeg0Ufbn7DdA+si9Qoxr68iS2PL0j0ei/ftZC iY/l9i84a4oJOxaSHwQIaQjot8cFBR2Z/zZXRKtbJ6+D8bNKKoaD91JYfzisB74MNwAR 2ZAvKiYG4SF0H2WaMVyg0MC3EsgN8eR395IsckgidIoPo4jke2cWeDTDtbnQyYwMCRSp IPv4dbhVsORCd8x1SovWN7d5IHLQSHzOMTCQQh59Z8Mm1T6UkIECXhPFlNh+BI9/X1zi y6V1+t6id2EfsWXZ86bArKfdfVctHF9p3zFJT/GxzxQ99KQKIy1iJX4t2f1eHgKJD0EP L+BQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=2dMQqpeyi0XKIpauyWT1oQVejUj1nmYGROEX25a4OR4=; b=CqAVvHnVbykRX00kQV/fsHGVXj9X0C+2Qv1OBUu62dGoKxc9Agef0bdn1ar8kn5p+z 70PWcUB3K7hNYUStDFGnhPE10aC+29DaocBwGnplUsxjq9LKwhpImWao2L3t+DLYbfRw RYEqunemEHvbC5ghehLaYleDpzONemhv3PgIsiMZDROqvLQGuo5xgjkLnbYJoxliGOmO 5Y7SsW4r6CJ3H5To/uHbjha3zHu1khd8o92WDPxC/MSF4MV2/E7RdFgO2RCf0Vb6eiOu jsEhA3Qb2C3Tl8GZKJJGrxmO8jg9C0hIvmB/Z4dVV5i2HpgBZpm39rCVcVP7T5bFMBSi yOKA== X-Gm-Message-State: AOAM532npcO6BjR5nA5Nu9YavYpNtqgJxlelYSnVbJJUatPyu7bmDvrR AsnODjElQHWe2fWlAlKh+YPGjnPM54s= X-Received: by 2002:a17:902:9f90:: with SMTP id g16mr20197123plq.146.1591146806315; Tue, 02 Jun 2020 18:13:26 -0700 (PDT) Received: from localhost.localdomain (174-21-143-238.tukw.qwest.net. [174.21.143.238]) by smtp.gmail.com with ESMTPSA id 3sm290067pfe.85.2020.06.02.18.13.25 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 02 Jun 2020 18:13:25 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v7 06/42] target/arm: Add DISAS_UPDATE_NOCHAIN Date: Tue, 2 Jun 2020 18:12:41 -0700 Message-Id: <20200603011317.473934-7-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200603011317.473934-1-richard.henderson@linaro.org> References: <20200603011317.473934-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::643; envelope-from=richard.henderson@linaro.org; helo=mail-pl1-x643.google.com X-detected-operating-system: by eggs.gnu.org: No matching host in p0f cache. That's all we know. X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: peter.maydell@linaro.org, qemu-arm@nongnu.org, steplong@quicinc.com Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" Add an option that writes back the PC, like DISAS_UPDATE_EXIT, but does not exit back to the main loop. Signed-off-by: Richard Henderson --- target/arm/translate.h | 2 ++ target/arm/translate-a64.c | 3 +++ target/arm/translate.c | 4 ++++ 3 files changed, 9 insertions(+) -- 2.25.1 Reviewed-by: Peter Maydell diff --git a/target/arm/translate.h b/target/arm/translate.h index 68a55251c3..81e47677c7 100644 --- a/target/arm/translate.h +++ b/target/arm/translate.h @@ -173,6 +173,8 @@ static inline void disas_set_insn_syndrome(DisasContext *s, uint32_t syn) * return from cpu_tb_exec. */ #define DISAS_EXIT DISAS_TARGET_9 +/* CPU state was modified dynamically; no need to exit, but do not chain. */ +#define DISAS_UPDATE_NOCHAIN DISAS_TARGET_10 #ifdef TARGET_AARCH64 void a64_translate_init(void); diff --git a/target/arm/translate-a64.c b/target/arm/translate-a64.c index 430f1867ab..d4d2f69d87 100644 --- a/target/arm/translate-a64.c +++ b/target/arm/translate-a64.c @@ -14340,6 +14340,9 @@ static void aarch64_tr_tb_stop(DisasContextBase *dcbase, CPUState *cpu) case DISAS_EXIT: tcg_gen_exit_tb(NULL, 0); break; + case DISAS_UPDATE_NOCHAIN: + gen_a64_set_pc_im(dc->base.pc_next); + /* fall through */ case DISAS_JUMP: tcg_gen_lookup_and_goto_ptr(); break; diff --git a/target/arm/translate.c b/target/arm/translate.c index 00a6fe6752..460be11f31 100644 --- a/target/arm/translate.c +++ b/target/arm/translate.c @@ -11085,6 +11085,7 @@ static void arm_tr_tb_stop(DisasContextBase *dcbase, CPUState *cpu) case DISAS_NEXT: case DISAS_TOO_MANY: case DISAS_UPDATE_EXIT: + case DISAS_UPDATE_NOCHAIN: gen_set_pc_im(dc, dc->base.pc_next); /* fall through */ default: @@ -11108,6 +11109,9 @@ static void arm_tr_tb_stop(DisasContextBase *dcbase, CPUState *cpu) case DISAS_TOO_MANY: gen_goto_tb(dc, 1, dc->base.pc_next); break; + case DISAS_UPDATE_NOCHAIN: + gen_set_pc_im(dc, dc->base.pc_next); + /* fall through */ case DISAS_JUMP: gen_goto_ptr(); break; From patchwork Wed Jun 3 01:12:42 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 187409 Delivered-To: patch@linaro.org Received: by 2002:a92:cf06:0:0:0:0:0 with SMTP id c6csp72159ilo; Tue, 2 Jun 2020 18:17:55 -0700 (PDT) X-Google-Smtp-Source: ABdhPJx9U/qY47iWBWW7llKcg4SZoH9W8SRD/D6xvSwgnHWm9kKap6orNc6IJuPpnpEZrfW4tRTr X-Received: by 2002:a5b:11:: with SMTP id a17mr458041ybp.290.1591147075552; Tue, 02 Jun 2020 18:17:55 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1591147075; cv=none; d=google.com; s=arc-20160816; b=AIK8kjh7lY5J1b0a4Lf65h4Q7GhfSgcZUS+KPQsYVAyIk+2BCALHCvJd0HQXi5ZqM+ FXiBle1nYqoyZowGEsr5Qxdy7ERxhBsQv08P+11tUCeQSkNEjLLCjT8l0YHG6zNJFlFv 8l425405jWtW64OY7e5cH6XjvDIyTy7qns1yWW835fweehiYPD47WuH7noALzN1YKB9F vWIOJ89FhS+MN498a5P2Yl04fxezi+8iMM0NrbIcBnSXeflsTok1NPoXYmGfgaQh61a6 uj89yY+tRKdBXPRJuE3RpxqfByVaRcKwNGlmU0EMsCwQLA56aJHGQDBxZyB7cUU0d7im QY4g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=03LJNoH/ukVkl3mS8L9CmNVy1qT1M4Av2oRp35Y7po4=; b=nNu2u5tCYuF4sZv8H25eRSMiF9YpcOjAhpXPPDDOByrcDxAvoZC4DEyGl9a4qvhuqj ZfvVsWKupZ14TypPW6kj45rKXfzvkFOhq6lHLA39pfwThd44azI+PjfQocZNKRYwLILg 6K9T7Pgv9i2dsdFb9KCPdIUwuXnocGYNgQvceMP4eDj1USY7Zk4irhpmwf1/Au5dlm8J sjrLQV/NX9sL43Sj7OX9j9HMpX/KB7MDkb6fwUlDOH/rDB76pAT5hBIEYx5ja6e47qkK pWIWqQg4XYOFKq0iXUqAMq627VbqPbZU//jwyZJA365fL/hXsxdqRaoAK32EuqmbQtT0 E+gA== ARC-Authentication-Results: i=1; mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=LTudRNbQ; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id t19si330578ybt.477.2020.06.02.18.17.55 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 02 Jun 2020 18:17:55 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=LTudRNbQ; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:35514 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jgI2U-0002s8-Vw for patch@linaro.org; Tue, 02 Jun 2020 21:17:55 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:48590) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jgHyE-0003ZJ-FL for qemu-devel@nongnu.org; Tue, 02 Jun 2020 21:13:30 -0400 Received: from mail-pf1-x443.google.com ([2607:f8b0:4864:20::443]:35105) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1jgHyD-0003XU-Cm for qemu-devel@nongnu.org; Tue, 02 Jun 2020 21:13:30 -0400 Received: by mail-pf1-x443.google.com with SMTP id h185so429910pfg.2 for ; Tue, 02 Jun 2020 18:13:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=03LJNoH/ukVkl3mS8L9CmNVy1qT1M4Av2oRp35Y7po4=; b=LTudRNbQwV+6JZ3antNWvutWhJSI6eVhnwYj31lmyuO7qaDahrCHjRsv8sQY0CCW71 iMxwHvcrv7F4lPWD/cpM8Qv7QVJznDNBmT0MkrAUm3PuwlsxpLhUrDJRdLEXP3ISoufr jNPX+x3aueQMhm60uQH6cKiNVFJuSDz9J98DDk/Zd5Nfwatzs/b9wTQ9Sfq9yWSrI93n 31fXyeYIxxIRPgkzVXgU0P87QNuXcyaqz4ECImFWBzpB9xPfJ+bccGvSqXYMmY7TTk8q UEqxd1kLGEjf1CUDjzXYB1L4PZs/HnUzCPaXdF7i9Z7xNILEVNZ9c+LunYc0YHvLbX35 A2VA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=03LJNoH/ukVkl3mS8L9CmNVy1qT1M4Av2oRp35Y7po4=; b=hA9624bx4QlQAQA6mtNQy78QVh5s5vTeTA4BBOA55oF0oSFc5FqIlZrntPay/WQ3RW DTFYVKhsTkI2iBOK4Nnl9kBONEWiBGa5MPRuvw4y+GiDtFJa99tmWCNjhjXM1wmSEMqL WvE571UfosYbxOv5nxJQb8s5ndRRJg8RgSFEhgMCo8+7wtXk6+vGWjR8Og8wQt1vtvpV 4PH6jG1Bnhx1OeKwG7RPLoVFLMTixtyWJzM41F/pBxF9QGCJsod3u5rPFCtIJ19c1ORj rvUjU2ogS6748WMP9MObmhvh8/snA1eC2tdegYPtj9dfxLeiYhHnbH9wA6QUMrIsZYiA Su2w== X-Gm-Message-State: AOAM530sIns2TyXJd/lgCEkwNwX8Fgp03KsPJjVwZ+Thal2rluZ3n0Av IwOAa0De84auYgUQLF+jcqwLxCJKEVw= X-Received: by 2002:a17:90a:950e:: with SMTP id t14mr2108841pjo.99.1591146807570; Tue, 02 Jun 2020 18:13:27 -0700 (PDT) Received: from localhost.localdomain (174-21-143-238.tukw.qwest.net. [174.21.143.238]) by smtp.gmail.com with ESMTPSA id 3sm290067pfe.85.2020.06.02.18.13.26 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 02 Jun 2020 18:13:26 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v7 07/42] target/arm: Add MTE system registers Date: Tue, 2 Jun 2020 18:12:42 -0700 Message-Id: <20200603011317.473934-8-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200603011317.473934-1-richard.henderson@linaro.org> References: <20200603011317.473934-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::443; envelope-from=richard.henderson@linaro.org; helo=mail-pf1-x443.google.com X-detected-operating-system: by eggs.gnu.org: No matching host in p0f cache. That's all we know. X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: peter.maydell@linaro.org, qemu-arm@nongnu.org, steplong@quicinc.com Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" This is TFSRE0_EL1, TFSR_EL1, TFSR_EL2, TFSR_EL3, RGSR_EL1, GCR_EL1, GMID_EL1, and PSTATE.TCO. Signed-off-by: Richard Henderson --- v3: Add GMID; add access_mte. v4: Define only TCO at mte_insn_reg. v6: Define RAZ/WI version of TCO at mte_insn_reg; honor TID5 for GMID_EL1; fix TFS crn/crm; recalc hflags after TCO. --- target/arm/cpu.h | 4 ++ target/arm/internals.h | 9 ++++ target/arm/helper.c | 94 ++++++++++++++++++++++++++++++++++++++ target/arm/translate-a64.c | 21 +++++++++ 4 files changed, 128 insertions(+) -- 2.25.1 Reviewed-by: Peter Maydell diff --git a/target/arm/cpu.h b/target/arm/cpu.h index f8ac11e73b..49cf37d43b 100644 --- a/target/arm/cpu.h +++ b/target/arm/cpu.h @@ -502,6 +502,9 @@ typedef struct CPUARMState { uint64_t pmccfiltr_el0; /* Performance Monitor Filter Register */ uint64_t vpidr_el2; /* Virtualization Processor ID Register */ uint64_t vmpidr_el2; /* Virtualization Multiprocessor ID Register */ + uint64_t tfsr_el[4]; /* tfsre0_el1 is index 0. */ + uint64_t gcr_el1; + uint64_t rgsr_el1; } cp15; struct { @@ -1282,6 +1285,7 @@ void pmu_init(ARMCPU *cpu); #define PSTATE_SS (1U << 21) #define PSTATE_PAN (1U << 22) #define PSTATE_UAO (1U << 23) +#define PSTATE_TCO (1U << 25) #define PSTATE_V (1U << 28) #define PSTATE_C (1U << 29) #define PSTATE_Z (1U << 30) diff --git a/target/arm/internals.h b/target/arm/internals.h index 4bdbc3a8ac..56b4672685 100644 --- a/target/arm/internals.h +++ b/target/arm/internals.h @@ -1159,6 +1159,9 @@ static inline uint32_t aarch64_pstate_valid_mask(const ARMISARegisters *id) if (isar_feature_aa64_uao(id)) { valid |= PSTATE_UAO; } + if (isar_feature_aa64_mte(id)) { + valid |= PSTATE_TCO; + } return valid; } @@ -1234,4 +1237,10 @@ void arm_log_exception(int idx); #endif /* !CONFIG_USER_ONLY */ +/* + * The log2 of the words in the tag block, for GMID_EL1.BS. + * The is the maximum, 256 bytes, which manipulates 64-bits of tags. + */ +#define GMID_EL1_BS 6 + #endif diff --git a/target/arm/helper.c b/target/arm/helper.c index f2ead07ead..88ca07ad0f 100644 --- a/target/arm/helper.c +++ b/target/arm/helper.c @@ -5879,6 +5879,9 @@ static void define_arm_vh_e2h_redirects_aliases(ARMCPU *cpu) { K(3, 0, 1, 2, 0), K(3, 4, 1, 2, 0), K(3, 5, 1, 2, 0), "ZCR_EL1", "ZCR_EL2", "ZCR_EL12", isar_feature_aa64_sve }, + { K(3, 0, 5, 6, 0), K(3, 4, 5, 6, 0), K(3, 5, 5, 6, 0), + "TFSR_EL1", "TFSR_EL2", "TFSR_EL12", isar_feature_aa64_mte }, + /* TODO: ARMv8.2-SPE -- PMSCR_EL2 */ /* TODO: ARMv8.4-Trace -- TRFCR_EL2 */ }; @@ -6853,6 +6856,86 @@ static const ARMCPRegInfo dcpodp_reg[] = { }; #endif /*CONFIG_USER_ONLY*/ +static CPAccessResult access_aa64_tid5(CPUARMState *env, const ARMCPRegInfo *ri, + bool isread) +{ + if ((arm_current_el(env) < 2) && (arm_hcr_el2_eff(env) & HCR_TID5)) { + return CP_ACCESS_TRAP_EL2; + } + + return CP_ACCESS_OK; +} + +static CPAccessResult access_mte(CPUARMState *env, const ARMCPRegInfo *ri, + bool isread) +{ + int el = arm_current_el(env); + + if (el < 2 && + arm_feature(env, ARM_FEATURE_EL2) && + !(arm_hcr_el2_eff(env) & HCR_ATA)) { + return CP_ACCESS_TRAP_EL2; + } + if (el < 3 && + arm_feature(env, ARM_FEATURE_EL3) && + !(env->cp15.scr_el3 & SCR_ATA)) { + return CP_ACCESS_TRAP_EL3; + } + return CP_ACCESS_OK; +} + +static uint64_t tco_read(CPUARMState *env, const ARMCPRegInfo *ri) +{ + return env->pstate & PSTATE_TCO; +} + +static void tco_write(CPUARMState *env, const ARMCPRegInfo *ri, uint64_t val) +{ + env->pstate = (env->pstate & ~PSTATE_TCO) | (val & PSTATE_TCO); +} + +static const ARMCPRegInfo mte_reginfo[] = { + { .name = "TFSRE0_EL1", .state = ARM_CP_STATE_AA64, + .opc0 = 3, .opc1 = 0, .crn = 5, .crm = 6, .opc2 = 1, + .access = PL1_RW, .accessfn = access_mte, + .fieldoffset = offsetof(CPUARMState, cp15.tfsr_el[0]) }, + { .name = "TFSR_EL1", .state = ARM_CP_STATE_AA64, + .opc0 = 3, .opc1 = 0, .crn = 5, .crm = 6, .opc2 = 0, + .access = PL1_RW, .accessfn = access_mte, + .fieldoffset = offsetof(CPUARMState, cp15.tfsr_el[1]) }, + { .name = "TFSR_EL2", .state = ARM_CP_STATE_AA64, + .opc0 = 3, .opc1 = 4, .crn = 5, .crm = 6, .opc2 = 0, + .access = PL2_RW, .accessfn = access_mte, + .fieldoffset = offsetof(CPUARMState, cp15.tfsr_el[2]) }, + { .name = "TFSR_EL3", .state = ARM_CP_STATE_AA64, + .opc0 = 3, .opc1 = 6, .crn = 5, .crm = 6, .opc2 = 0, + .access = PL3_RW, + .fieldoffset = offsetof(CPUARMState, cp15.tfsr_el[3]) }, + { .name = "RGSR_EL1", .state = ARM_CP_STATE_AA64, + .opc0 = 3, .opc1 = 0, .crn = 1, .crm = 0, .opc2 = 5, + .access = PL1_RW, .accessfn = access_mte, + .fieldoffset = offsetof(CPUARMState, cp15.rgsr_el1) }, + { .name = "GCR_EL1", .state = ARM_CP_STATE_AA64, + .opc0 = 3, .opc1 = 0, .crn = 1, .crm = 0, .opc2 = 6, + .access = PL1_RW, .accessfn = access_mte, + .fieldoffset = offsetof(CPUARMState, cp15.gcr_el1) }, + { .name = "GMID_EL1", .state = ARM_CP_STATE_AA64, + .opc0 = 3, .opc1 = 1, .crn = 0, .crm = 0, .opc2 = 4, + .access = PL1_R, .accessfn = access_aa64_tid5, + .type = ARM_CP_CONST, .resetvalue = GMID_EL1_BS }, + { .name = "TCO", .state = ARM_CP_STATE_AA64, + .opc0 = 3, .opc1 = 3, .crn = 4, .crm = 2, .opc2 = 7, + .type = ARM_CP_NO_RAW, + .access = PL0_RW, .readfn = tco_read, .writefn = tco_write }, + REGINFO_SENTINEL +}; + +static const ARMCPRegInfo mte_tco_ro_reginfo[] = { + { .name = "TCO", .state = ARM_CP_STATE_AA64, + .opc0 = 3, .opc1 = 3, .crn = 4, .crm = 2, .opc2 = 7, + .type = ARM_CP_CONST, .access = PL0_RW, }, + REGINFO_SENTINEL +}; #endif static CPAccessResult access_predinv(CPUARMState *env, const ARMCPRegInfo *ri, @@ -7978,6 +8061,17 @@ void register_cp_regs_for_features(ARMCPU *cpu) } } #endif /*CONFIG_USER_ONLY*/ + + /* + * If full MTE is enabled, add all of the system registers. + * If only "instructions available at EL0" are enabled, + * then define only a RAZ/WI version of PSTATE.TCO. + */ + if (cpu_isar_feature(aa64_mte, cpu)) { + define_arm_cp_regs(cpu, mte_reginfo); + } else if (cpu_isar_feature(aa64_mte_insn_reg, cpu)) { + define_arm_cp_regs(cpu, mte_tco_ro_reginfo); + } #endif if (cpu_isar_feature(any_predinv, cpu)) { diff --git a/target/arm/translate-a64.c b/target/arm/translate-a64.c index d4d2f69d87..284266be67 100644 --- a/target/arm/translate-a64.c +++ b/target/arm/translate-a64.c @@ -1610,6 +1610,27 @@ static void handle_msr_i(DisasContext *s, uint32_t insn, s->base.is_jmp = DISAS_UPDATE_EXIT; break; + case 0x1c: /* TCO */ + if (dc_isar_feature(aa64_mte, s)) { + /* Full MTE is enabled -- set the TCO bit as directed. */ + if (crm & 1) { + set_pstate_bits(PSTATE_TCO); + } else { + clear_pstate_bits(PSTATE_TCO); + } + t1 = tcg_const_i32(s->current_el); + gen_helper_rebuild_hflags_a64(cpu_env, t1); + tcg_temp_free_i32(t1); + /* Many factors, including TCO, go into MTE_ACTIVE. */ + s->base.is_jmp = DISAS_UPDATE_NOCHAIN; + } else if (dc_isar_feature(aa64_mte_insn_reg, s)) { + /* Only "instructions accessible at EL0" -- PSTATE.TCO is WI. */ + s->base.is_jmp = DISAS_NEXT; + } else { + goto do_unallocated; + } + break; + default: do_unallocated: unallocated_encoding(s); From patchwork Wed Jun 3 01:12:43 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 187408 Delivered-To: patch@linaro.org Received: by 2002:a92:cf06:0:0:0:0:0 with SMTP id c6csp71517ilo; Tue, 2 Jun 2020 18:16:52 -0700 (PDT) X-Google-Smtp-Source: ABdhPJyi/uG7FdPWqF8xvqvJnQx6KKfFcWcVZFqh45pWQ39xJ7RgyMtF8VkGiKftlfpWVKofMBUu X-Received: by 2002:a25:ba81:: with SMTP id s1mr45035852ybg.508.1591147012368; Tue, 02 Jun 2020 18:16:52 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1591147012; cv=none; d=google.com; s=arc-20160816; b=w8SttrcySLZ5UUKSKthsCVHXzLPFHWx+SfQ0S7bvv08BliPnepHxqHPzFrLmC78hO9 lmErNdOfhhff4uR9liYB40ESa3Zo+IxN9vB4Dd543kK+w+r4Dmi/mxv4wCtoCFyplEdf cT7MKBYt/qC7SlReSfIiHSWK2aVkUsjsmjShgcoU7PJkumzSQbhnmlsj22xbiELaHm8W zs1vd+tvYhvkd/D+R3AjYGume0aqFm0UZ6/jHGIoKxyTOHKAf84xN6AIwhechupeNrt4 8saIcdpfmRF0K1uN5yebU/AsDrfHH+y7dINcrTqqnNWFuB3Uk7L6/68pLDq2s0hHY/DD Vcsw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=Daiw2EtJyD/dc/N0oqXfOg3YFjBed/k9utJsMidR3kc=; b=XtAxVGdTSzCMHdt4UWENBmamjyKxq9G0GS9cWMd/ESXhU2K5GPCgJG2Dxea+YCRaBf 0pV7AneshZS07+oNkXu4K3XbdMpC/WTTTdO0OJYzwS7VFio6wzz/eiY9vyh2wL8ZexVD EjtzHQNTBSmRvU65GeJi8ZLWEt/QjPRg0li3l3qwHeH0E9Dbn+TS5XWru7Y14RC+AyXn 37NahaZS99rEHxSJevSJ6ayAI2ca9vWURQWRnH0VBqTKQPOxGnm19xEWdY6jMhV63M9O 2wmQPFgHG5K44faR1Fy+vNytm/9bMQeyifZBca78QndbxF2d3eviS3U2J8IQJWiVv5B0 RCqw== ARC-Authentication-Results: i=1; mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=DsCrpmOi; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id x9si321497ybp.92.2020.06.02.18.16.52 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 02 Jun 2020 18:16:52 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=DsCrpmOi; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:56780 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jgI1T-0008V6-NH for patch@linaro.org; Tue, 02 Jun 2020 21:16:51 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:48602) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jgHyF-0003dP-Ng for qemu-devel@nongnu.org; Tue, 02 Jun 2020 21:13:31 -0400 Received: from mail-pg1-x544.google.com ([2607:f8b0:4864:20::544]:35916) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1jgHyE-0003Xl-Nc for qemu-devel@nongnu.org; Tue, 02 Jun 2020 21:13:31 -0400 Received: by mail-pg1-x544.google.com with SMTP id t7so571966pgt.3 for ; Tue, 02 Jun 2020 18:13:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=Daiw2EtJyD/dc/N0oqXfOg3YFjBed/k9utJsMidR3kc=; b=DsCrpmOiVKG2vmtyDCwQJQQOthyPaVs60oHTNNtknbRuSLJ+9M+c8gHavPCM7zh0K6 n1yvMYwkPw0VpG0C5sk7F5GoegE2ZSkZnmVoBG9VzoiG2WfcBo1YoFPRw6KLVzY2inPt VPoeOTV/jWUbBhPojkRqPrqTnyUDLAo8Lz5biVacTlk+r96zZWhKQtHvgwz7xAhzmYlk WsW4ny5ixbHCCCi9xi5/SoJGOoiZtqO0aONRJkiwSVdbiF8jUtPZ7iXbC1tKaXjF4yte DDONKSc0tuCuHnGV7Wy+CPTPwXaP8Bv4ZtwidCz3gCxe2+/35hsmYqiHBZ5bV/75RRkC 0sFQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=Daiw2EtJyD/dc/N0oqXfOg3YFjBed/k9utJsMidR3kc=; b=LnAyZ3jDnrHNtiaAwEzx4lnEmn3FCxorespuUvHc5d11eNaZ4Z8rxktVke3PeVzae1 WJxgIV3xTtRsGSFaeaat59CCuCttsnAEjXhLkGcdleHG0gPGL/OfxyWXRRpmpuXKoZZq aNLvi2GlEUjzo8Y5/a0rFRiStBXovwzl6iEVoAZs40wtwMW3NXf9Bk6T5L7jeAW0+O09 P/e2oaJ7GPS8wVcD9qsW71WBUtfoJL2zqiqH3XgxT337t/gwEsiarDLjnHoyUPVJkF3d SXVy7YIwtQBcuvR4UVUKL0wghjzDmFw6jt6ZbFcY1bGpWI+xbXPl7Q86Zn1EOGelHr6A lvVg== X-Gm-Message-State: AOAM530RzxgLQhuFESPaUhpy23PTtrUaaeQ1YbIoYFLuioyDvSvpXB2o FNtNGr4JyZtbk45+4OR2CuXLyMkwhbU= X-Received: by 2002:a65:6790:: with SMTP id e16mr27471091pgr.145.1591146808805; Tue, 02 Jun 2020 18:13:28 -0700 (PDT) Received: from localhost.localdomain (174-21-143-238.tukw.qwest.net. [174.21.143.238]) by smtp.gmail.com with ESMTPSA id 3sm290067pfe.85.2020.06.02.18.13.27 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 02 Jun 2020 18:13:28 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v7 08/42] target/arm: Add MTE bits to tb_flags Date: Tue, 2 Jun 2020 18:12:43 -0700 Message-Id: <20200603011317.473934-9-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200603011317.473934-1-richard.henderson@linaro.org> References: <20200603011317.473934-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::544; envelope-from=richard.henderson@linaro.org; helo=mail-pg1-x544.google.com X-detected-operating-system: by eggs.gnu.org: No matching host in p0f cache. That's all we know. X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: peter.maydell@linaro.org, qemu-arm@nongnu.org, steplong@quicinc.com Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" Cache the composite ATA setting. Cache when MTE is fully enabled, i.e. access to tags are enabled and tag checks affect the PE. Do this for both the normal context and the UNPRIV context. Signed-off-by: Richard Henderson --- v3: Remove stub helper_mte_check; moved to a later patch. v6: Add mte0_active and ata bits; drop reviewed-by. --- target/arm/cpu.h | 12 ++++++++---- target/arm/internals.h | 18 +++++++++++++++++ target/arm/translate.h | 5 +++++ target/arm/helper.c | 40 ++++++++++++++++++++++++++++++++++++++ target/arm/translate-a64.c | 4 ++++ 5 files changed, 75 insertions(+), 4 deletions(-) -- 2.25.1 Reviewed-by: Peter Maydell diff --git a/target/arm/cpu.h b/target/arm/cpu.h index 49cf37d43b..a5d3b6c9ee 100644 --- a/target/arm/cpu.h +++ b/target/arm/cpu.h @@ -3187,10 +3187,10 @@ typedef ARMCPU ArchCPU; * | | | TBFLAG_A32 | | * | | +-----+----------+ TBFLAG_AM32 | * | TBFLAG_ANY | |TBFLAG_M32| | - * | | +-+----------+--------------| - * | | | TBFLAG_A64 | - * +--------------+---------+---------------------------+ - * 31 20 15 0 + * | +-----------+----------+--------------| + * | | TBFLAG_A64 | + * +--------------+-------------------------------------+ + * 31 20 0 * * Unless otherwise noted, these bits are cached in env->hflags. */ @@ -3257,6 +3257,10 @@ FIELD(TBFLAG_A64, BT, 9, 1) FIELD(TBFLAG_A64, BTYPE, 10, 2) /* Not cached. */ FIELD(TBFLAG_A64, TBID, 12, 2) FIELD(TBFLAG_A64, UNPRIV, 14, 1) +FIELD(TBFLAG_A64, ATA, 15, 1) +FIELD(TBFLAG_A64, TCMA, 16, 2) +FIELD(TBFLAG_A64, MTE_ACTIVE, 18, 1) +FIELD(TBFLAG_A64, MTE0_ACTIVE, 19, 1) /** * cpu_mmu_index: diff --git a/target/arm/internals.h b/target/arm/internals.h index 56b4672685..53e249687b 100644 --- a/target/arm/internals.h +++ b/target/arm/internals.h @@ -1198,6 +1198,24 @@ static inline int exception_target_el(CPUARMState *env) return target_el; } +/* Determine if allocation tags are available. */ +static inline bool allocation_tag_access_enabled(CPUARMState *env, int el, + uint64_t sctlr) +{ + if (el < 3 + && arm_feature(env, ARM_FEATURE_EL3) + && !(env->cp15.scr_el3 & SCR_ATA)) { + return false; + } + if (el < 2 + && arm_feature(env, ARM_FEATURE_EL2) + && !(arm_hcr_el2_eff(env) & HCR_ATA)) { + return false; + } + sctlr &= (el == 0 ? SCTLR_ATA0 : SCTLR_ATA); + return sctlr != 0; +} + #ifndef CONFIG_USER_ONLY /* Security attributes for an address, as returned by v8m_security_lookup. */ diff --git a/target/arm/translate.h b/target/arm/translate.h index 81e47677c7..4fb0feece9 100644 --- a/target/arm/translate.h +++ b/target/arm/translate.h @@ -30,6 +30,7 @@ typedef struct DisasContext { ARMMMUIdx mmu_idx; /* MMU index to use for normal loads/stores */ uint8_t tbii; /* TBI1|TBI0 for insns */ uint8_t tbid; /* TBI1|TBI0 for data */ + uint8_t tcma; /* TCMA1|TCMA0 for MTE */ bool ns; /* Use non-secure CPREG bank on access */ int fp_excp_el; /* FP exception EL or 0 if enabled */ int sve_excp_el; /* SVE exception EL or 0 if enabled */ @@ -77,6 +78,10 @@ typedef struct DisasContext { bool unpriv; /* True if v8.3-PAuth is active. */ bool pauth_active; + /* True if v8.5-MTE access to tags is enabled. */ + bool ata; + /* True if v8.5-MTE tag checks affect the PE; index with is_unpriv. */ + bool mte_active[2]; /* True with v8.5-BTI and SCTLR_ELx.BT* set. */ bool bt; /* True if any CP15 access is trapped by HSTR_EL2 */ diff --git a/target/arm/helper.c b/target/arm/helper.c index 88ca07ad0f..8aaa6f22b2 100644 --- a/target/arm/helper.c +++ b/target/arm/helper.c @@ -10653,6 +10653,16 @@ static int aa64_va_parameter_tbid(uint64_t tcr, ARMMMUIdx mmu_idx) } } +static int aa64_va_parameter_tcma(uint64_t tcr, ARMMMUIdx mmu_idx) +{ + if (regime_has_2_ranges(mmu_idx)) { + return extract64(tcr, 57, 2); + } else { + /* Replicate the single TCMA bit so we always have 2 bits. */ + return extract32(tcr, 30, 1) * 3; + } +} + ARMVAParameters aa64_va_parameters(CPUARMState *env, uint64_t va, ARMMMUIdx mmu_idx, bool data) { @@ -12677,6 +12687,36 @@ static uint32_t rebuild_hflags_a64(CPUARMState *env, int el, int fp_el, } } + if (cpu_isar_feature(aa64_mte, env_archcpu(env))) { + /* + * Set MTE_ACTIVE if any access may be Checked, and leave clear + * if all accesses must be Unchecked: + * 1) If no TBI, then there are no tags in the address to check, + * 2) If Tag Check Override, then all accesses are Unchecked, + * 3) If Tag Check Fail == 0, then Checked access have no effect, + * 4) If no Allocation Tag Access, then all accesses are Unchecked. + */ + if (allocation_tag_access_enabled(env, el, sctlr)) { + flags = FIELD_DP32(flags, TBFLAG_A64, ATA, 1); + if (tbid + && !(env->pstate & PSTATE_TCO) + && (sctlr & (el == 0 ? SCTLR_TCF0 : SCTLR_TCF))) { + flags = FIELD_DP32(flags, TBFLAG_A64, MTE_ACTIVE, 1); + } + } + /* And again for unprivileged accesses, if required. */ + if (FIELD_EX32(flags, TBFLAG_A64, UNPRIV) + && tbid + && !(env->pstate & PSTATE_TCO) + && (sctlr & SCTLR_TCF0) + && allocation_tag_access_enabled(env, 0, sctlr)) { + flags = FIELD_DP32(flags, TBFLAG_A64, MTE0_ACTIVE, 1); + } + /* Cache TCMA as well as TBI. */ + flags = FIELD_DP32(flags, TBFLAG_A64, TCMA, + aa64_va_parameter_tcma(tcr, mmu_idx)); + } + return rebuild_hflags_common(env, fp_el, mmu_idx, flags); } diff --git a/target/arm/translate-a64.c b/target/arm/translate-a64.c index 284266be67..ec2013782d 100644 --- a/target/arm/translate-a64.c +++ b/target/arm/translate-a64.c @@ -14213,6 +14213,7 @@ static void aarch64_tr_init_disas_context(DisasContextBase *dcbase, dc->mmu_idx = core_to_aa64_mmu_idx(core_mmu_idx); dc->tbii = FIELD_EX32(tb_flags, TBFLAG_A64, TBII); dc->tbid = FIELD_EX32(tb_flags, TBFLAG_A64, TBID); + dc->tcma = FIELD_EX32(tb_flags, TBFLAG_A64, TCMA); dc->current_el = arm_mmu_idx_to_el(dc->mmu_idx); #if !defined(CONFIG_USER_ONLY) dc->user = (dc->current_el == 0); @@ -14224,6 +14225,9 @@ static void aarch64_tr_init_disas_context(DisasContextBase *dcbase, dc->bt = FIELD_EX32(tb_flags, TBFLAG_A64, BT); dc->btype = FIELD_EX32(tb_flags, TBFLAG_A64, BTYPE); dc->unpriv = FIELD_EX32(tb_flags, TBFLAG_A64, UNPRIV); + dc->ata = FIELD_EX32(tb_flags, TBFLAG_A64, ATA); + dc->mte_active[0] = FIELD_EX32(tb_flags, TBFLAG_A64, MTE_ACTIVE); + dc->mte_active[1] = FIELD_EX32(tb_flags, TBFLAG_A64, MTE0_ACTIVE); dc->vec_len = 0; dc->vec_stride = 0; dc->cp_regs = arm_cpu->cp_regs; From patchwork Wed Jun 3 01:12:44 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 187415 Delivered-To: patch@linaro.org Received: by 2002:a92:cf06:0:0:0:0:0 with SMTP id c6csp73540ilo; Tue, 2 Jun 2020 18:20:16 -0700 (PDT) X-Google-Smtp-Source: ABdhPJzLRxXF9woA4stKDfn6QBa1HwE2AMHDhryXz+mkQlzftIAVHSlzE9tAko7eY8r+Mp+eOhj+ X-Received: by 2002:a25:c103:: with SMTP id r3mr42493149ybf.343.1591147216073; Tue, 02 Jun 2020 18:20:16 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1591147216; cv=none; d=google.com; s=arc-20160816; b=K9tyut3MYsdlHqZzmAoH8sd1unsvSij3Bexegy42mnFYm3oz8O6J87Bf92jIURBlb5 ZRIoGUlYDsAOELuaVIk6MjjQzcoOIEtgdND7F0YKcf3rc51i0dnkIdXVfdBIuOFvK5zK 43DQb6ZZw1ialaSPUb8AkkNzcAN7CdW8R7Zfk7aFRUcYsrhcWg6HUaHDZH2F0Vp0BWoq AodKo3rfH62cg2eo+2ixlmfbt7YlbpRkLjJmMe5sXdStHVpKG1ltzAJwH1/92uiqgTn6 1BIbyB4c3WXeUVBua4Jybbs6ap0YxTwAHvbBBUot815phgv5C3yJcuco0FHEcMrZHSLR U1mw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=GhH3aRALfbOeqG40MvYbNczfufYSgHXrHQfwMMscNhw=; b=bLWowyr0KjR7wFS676+U3nfxs2jTXQkzRwHFU6LH6sLTlr/VZSr3gBhX5T+5OG5yfS r/5n4qUd3ocvdpLzFjPYD6qaLEyZeJEf8uH0avbuI/vZjRPl57sQROE7bKgr7j20VgId Cdc/kmt6N2r9ENX/tWBbLVHQ2qMpScnSc1H9JGR4m/2KpmzapT8rK0O2ebl0f6V3LvS0 wPsL4wNb35Zcgg+K2Pxe9LaiSrTXRQmkuN40U9BKyaDKa+hCMEbFsxnsQB3aHd93tACG S5vWwvjduyj1NgQeARxCKxE7hTXfX3inQLIobjYCwVII4tYour6q9nb2JiB2lIjrX2ky xObA== ARC-Authentication-Results: i=1; mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=vdaW0Hce; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id d25si320148yba.256.2020.06.02.18.20.15 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 02 Jun 2020 18:20:16 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=vdaW0Hce; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:46188 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jgI4l-00076F-Dm for patch@linaro.org; Tue, 02 Jun 2020 21:20:15 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:48610) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jgHyG-0003gq-Us for qemu-devel@nongnu.org; Tue, 02 Jun 2020 21:13:32 -0400 Received: from mail-pl1-x643.google.com ([2607:f8b0:4864:20::643]:36970) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1jgHyF-0003Y1-VM for qemu-devel@nongnu.org; Tue, 02 Jun 2020 21:13:32 -0400 Received: by mail-pl1-x643.google.com with SMTP id y18so201799plr.4 for ; Tue, 02 Jun 2020 18:13:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=GhH3aRALfbOeqG40MvYbNczfufYSgHXrHQfwMMscNhw=; b=vdaW0Hcels1M2Zq9beZmzpEqhiEFPjFw30z7uAwoOk5TdlRoKypGt3YjaQKlwyExuO 13McAgFS9Z4COlbTv9r5B0DZmOtQW9/R22/UmerUXZQJXK3hhXvm86qsvmzI3jXavAzI bKmHLRhIVzG/OODNFNyK9s510s1QpV/NmTd+nI2sj3Okk42xQwClEqa/Ge1FRFHC5nyD 7OBzMWe1+Iw78s8CrNNlxyKqzUwE6EC6Tsc44v37n3EpwHFwNXOlNujJiwgeUK40YQcy oot2w0We8uclqJuMivQDu7z2UkXWUdqLfjvMM4lHlSpUP8xf1TVJEr1Vp2PIXdD5bFv7 7oSg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=GhH3aRALfbOeqG40MvYbNczfufYSgHXrHQfwMMscNhw=; b=nOMuwCBChftZgI5G7Y/LK9Gm1R1gVBIznC9rBUA9AYwYWaVOP8bHSvolfj7UQzG4ol u5aA+XzCfYCklLWNoRvwXzP64YiV1N1QCymT9n5wsos5abulZYwKQa3WMLZg9ux7CNyy VuLMxizNbeSlhAHejW3aVy0zN3k8ctLYI7eUqkvNBc21sXAuRxt1O+FJg8NyfZMl6+bd yyseisBV6G5B55wd/38HvqakMp5diBybtNmUkPpId4Afb1UujXaS67dEjWw2DCWjPGdN nQ/M+9IiAWBDrONvHPf/UAHOPplNvayZWzIU5ZksLRNmbPzza/GYhuhqIb+1QYUnZpSH 7QOw== X-Gm-Message-State: AOAM532u5mtohw+GmwNy8Yt+DXASh4yxIYKjBHrn0xjLYdadyaqlZGjI e8upK8xYeFt4f3d3dukvr/ktzq+6/CQ= X-Received: by 2002:a17:902:b185:: with SMTP id s5mr28116718plr.304.1591146810187; Tue, 02 Jun 2020 18:13:30 -0700 (PDT) Received: from localhost.localdomain (174-21-143-238.tukw.qwest.net. [174.21.143.238]) by smtp.gmail.com with ESMTPSA id 3sm290067pfe.85.2020.06.02.18.13.29 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 02 Jun 2020 18:13:29 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v7 09/42] target/arm: Implement the IRG instruction Date: Tue, 2 Jun 2020 18:12:44 -0700 Message-Id: <20200603011317.473934-10-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200603011317.473934-1-richard.henderson@linaro.org> References: <20200603011317.473934-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::643; envelope-from=richard.henderson@linaro.org; helo=mail-pl1-x643.google.com X-detected-operating-system: by eggs.gnu.org: No matching host in p0f cache. That's all we know. X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: peter.maydell@linaro.org, qemu-arm@nongnu.org, steplong@quicinc.com Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" Signed-off-by: Richard Henderson --- v2: Update to 00eac5. Merge choose_random_nonexcluded_tag into helper_irg since that pseudo function no longer exists separately. v6: Remove obsolete logical/physical tag distinction; implement inline for !ATA. --- target/arm/helper-a64.h | 2 ++ target/arm/internals.h | 5 +++ target/arm/mte_helper.c | 72 ++++++++++++++++++++++++++++++++++++++ target/arm/translate-a64.c | 18 ++++++++++ target/arm/Makefile.objs | 1 + 5 files changed, 98 insertions(+) create mode 100644 target/arm/mte_helper.c -- 2.25.1 Reviewed-by: Peter Maydell diff --git a/target/arm/helper-a64.h b/target/arm/helper-a64.h index 3df7c185aa..587ccbe42f 100644 --- a/target/arm/helper-a64.h +++ b/target/arm/helper-a64.h @@ -103,3 +103,5 @@ DEF_HELPER_FLAGS_3(autda, TCG_CALL_NO_WG, i64, env, i64, i64) DEF_HELPER_FLAGS_3(autdb, TCG_CALL_NO_WG, i64, env, i64, i64) DEF_HELPER_FLAGS_2(xpaci, TCG_CALL_NO_RWG_SE, i64, env, i64) DEF_HELPER_FLAGS_2(xpacd, TCG_CALL_NO_RWG_SE, i64, env, i64) + +DEF_HELPER_FLAGS_3(irg, TCG_CALL_NO_RWG, i64, env, i64, i64) diff --git a/target/arm/internals.h b/target/arm/internals.h index 53e249687b..ae611a6ff5 100644 --- a/target/arm/internals.h +++ b/target/arm/internals.h @@ -1261,4 +1261,9 @@ void arm_log_exception(int idx); */ #define GMID_EL1_BS 6 +static inline uint64_t address_with_allocation_tag(uint64_t ptr, int rtag) +{ + return deposit64(ptr, 56, 4, rtag); +} + #endif diff --git a/target/arm/mte_helper.c b/target/arm/mte_helper.c new file mode 100644 index 0000000000..539a04de84 --- /dev/null +++ b/target/arm/mte_helper.c @@ -0,0 +1,72 @@ +/* + * ARM v8.5-MemTag Operations + * + * Copyright (c) 2020 Linaro, Ltd. + * + * This library is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2.1 of the License, or (at your option) any later version. + * + * This library is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with this library; if not, see . + */ + +#include "qemu/osdep.h" +#include "cpu.h" +#include "internals.h" +#include "exec/exec-all.h" +#include "exec/cpu_ldst.h" +#include "exec/helper-proto.h" + + +static int choose_nonexcluded_tag(int tag, int offset, uint16_t exclude) +{ + if (exclude == 0xffff) { + return 0; + } + if (offset == 0) { + while (exclude & (1 << tag)) { + tag = (tag + 1) & 15; + } + } else { + do { + do { + tag = (tag + 1) & 15; + } while (exclude & (1 << tag)); + } while (--offset > 0); + } + return tag; +} + +uint64_t HELPER(irg)(CPUARMState *env, uint64_t rn, uint64_t rm) +{ + int rtag; + + /* + * Our IMPDEF choice for GCR_EL1.RRND==1 is to behave as if + * GCR_EL1.RRND==0, always producing deterministic results. + */ + uint16_t exclude = extract32(rm | env->cp15.gcr_el1, 0, 16); + int start = extract32(env->cp15.rgsr_el1, 0, 4); + int seed = extract32(env->cp15.rgsr_el1, 8, 16); + int offset, i; + + /* RandomTag */ + for (i = offset = 0; i < 4; ++i) { + /* NextRandomTagBit */ + int top = (extract32(seed, 5, 1) ^ extract32(seed, 3, 1) ^ + extract32(seed, 2, 1) ^ extract32(seed, 0, 1)); + seed = (top << 15) | (seed >> 1); + offset |= top << i; + } + rtag = choose_nonexcluded_tag(start, offset, exclude); + env->cp15.rgsr_el1 = rtag | (seed << 8); + + return address_with_allocation_tag(rn, rtag); +} diff --git a/target/arm/translate-a64.c b/target/arm/translate-a64.c index ec2013782d..2481561925 100644 --- a/target/arm/translate-a64.c +++ b/target/arm/translate-a64.c @@ -226,6 +226,12 @@ static TCGv_i64 clean_data_tbi(DisasContext *s, TCGv_i64 addr) return clean; } +/* Insert a zero tag into src, with the result at dst. */ +static void gen_address_with_allocation_tag0(TCGv_i64 dst, TCGv_i64 src) +{ + tcg_gen_andi_i64(dst, src, ~MAKE_64BIT_MASK(56, 4)); +} + typedef struct DisasCompare64 { TCGCond cond; TCGv_i64 value; @@ -5275,6 +5281,18 @@ static void disas_data_proc_2src(DisasContext *s, uint32_t insn) case 3: /* SDIV */ handle_div(s, true, sf, rm, rn, rd); break; + case 4: /* IRG */ + if (sf == 0 || !dc_isar_feature(aa64_mte_insn_reg, s)) { + goto do_unallocated; + } + if (s->ata) { + gen_helper_irg(cpu_reg_sp(s, rd), cpu_env, + cpu_reg_sp(s, rn), cpu_reg(s, rm)); + } else { + gen_address_with_allocation_tag0(cpu_reg_sp(s, rd), + cpu_reg_sp(s, rn)); + } + break; case 8: /* LSLV */ handle_shift_reg(s, A64_SHIFT_TYPE_LSL, sf, rm, rn, rd); break; diff --git a/target/arm/Makefile.objs b/target/arm/Makefile.objs index 83febd232c..fa39fd7c83 100644 --- a/target/arm/Makefile.objs +++ b/target/arm/Makefile.objs @@ -86,3 +86,4 @@ obj-$(CONFIG_SOFTMMU) += psci.o obj-$(TARGET_AARCH64) += translate-a64.o helper-a64.o obj-$(TARGET_AARCH64) += translate-sve.o sve_helper.o obj-$(TARGET_AARCH64) += pauth_helper.o +obj-$(TARGET_AARCH64) += mte_helper.o From patchwork Wed Jun 3 01:12:45 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 187411 Delivered-To: patch@linaro.org Received: by 2002:a92:cf06:0:0:0:0:0 with SMTP id c6csp72569ilo; Tue, 2 Jun 2020 18:18:41 -0700 (PDT) X-Google-Smtp-Source: ABdhPJyUWXKvqH6SV2hBpqboHg6tkNLVwNWn++miKwxnz9Ygm7vjqZ053gn1agvQKFcMDzR01OZ5 X-Received: by 2002:a25:e86:: with SMTP id 128mr49231487ybo.344.1591147121151; Tue, 02 Jun 2020 18:18:41 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1591147121; cv=none; d=google.com; s=arc-20160816; b=SeR3DjTjc8AB/G2/olyGLJJ48T5f1m4GEvbFBqY3LZaV+zbxtaDZ3RF0qnOvqqy8ZM 65h6DaN03UYASWyn8tVm4bDUncbw5OpiBjsMA1twP4t8g9/bKAotJyyQ0rPcF5/C2npL FH1AyLPBjh+7dL4WOv5xCW7yen1F+M7/DMNmUTnnaratQnTJ5QgPZ+Le5Y5cKRQVum4H FM9hGa1b8ZpUTVpYU8P9JaFWOCOgGGcDKBX5+JcDK/qOuJqUUAVQxuFKtElwyNqDnwwX 3SJTE9un745ZFbT2orM/AWSd18h+3U6vd7Gm5SkZnWT5PplFSZ1kTCcw9BmRzF8VGzNk 2skw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=MehLLPs7OH2d6Ox8Z4/DEtryQBHArcvcM5ew1vNvLQQ=; b=g1JovDt27ZnF7JIcKiLWEA3y5O+lmVQBvhIA90QZX5/X3aZcAwcNdYiLDyXcVWxQLp n6J8wsm94bkSWJj+lrDMasA2XZ/1Ee0uieAo/vFDBxOFLlMo/lG6GPrKJbR1AKW2p5Xu 1g9j+grPK4WEAleT231mZzesxGwmU/E0cCpTxpqrpMq1jrXaOx1hyoFrvpZcTTR0bhka VGgO1s4Xavwk8PDofjR7HXm6N+1irYAaNXeMFGks9UlAQC4YuSvCZqp4SCrzxtBIDkRF XV/s8bs160hoMSW9cDEzH+eaHJGb6vhY8PUsxqkdoUxEVohL7X29l1zIu0hYePPM2YUF g/yw== ARC-Authentication-Results: i=1; mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=Ak3KhUJg; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id c2si358143yba.168.2020.06.02.18.18.41 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 02 Jun 2020 18:18:41 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=Ak3KhUJg; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:40214 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jgI3D-0004l0-LK for patch@linaro.org; Tue, 02 Jun 2020 21:18:40 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:48622) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jgHyI-0003kN-4y for qemu-devel@nongnu.org; Tue, 02 Jun 2020 21:13:34 -0400 Received: from mail-pl1-x644.google.com ([2607:f8b0:4864:20::644]:36971) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1jgHyH-0003YG-9u for qemu-devel@nongnu.org; Tue, 02 Jun 2020 21:13:33 -0400 Received: by mail-pl1-x644.google.com with SMTP id y18so201814plr.4 for ; Tue, 02 Jun 2020 18:13:32 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=MehLLPs7OH2d6Ox8Z4/DEtryQBHArcvcM5ew1vNvLQQ=; b=Ak3KhUJginINoFqCHKHBtNmOVSbXtdWbSHFc0M1UYoiTOC5/tqwWY1FkxRiDFgM0g/ nNQR8uL2qIYdvwFc0jFA1UrIOGLJcuWfbcSAnIJ2gwJiktPHS6juDqM02DWoRAHKDE1u AtOpxZcdN7IT4KCUWafayECCLuF1mQH9dizLahlIEh8Ddqp1r3uchxVp4OLn84RkcRD+ kXpCf1J6pETkqiyT3V+Bk+whJp2GcEZcFYI2X9vW8Gge52oj1+LNpzmGZHRX4ILk6PUw CcG49UQuj1s4keYlrlCETsEeunqnai8iGGZwYPTKxaY7hBcHgazlbxqAVsmluMBW+RWU 6n1A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=MehLLPs7OH2d6Ox8Z4/DEtryQBHArcvcM5ew1vNvLQQ=; b=L6DZ/6xgU34Q2Hg7mpGXQuD89kvYZ7SW0YXgTOCbuhx+T262X7ZGZcXqmKT7DgetGN SJ8UfnT7XqoCRUFkctok2WyOJpvgORLTU6rm1BMrtPxp6tWkuQDxOWaI6Jeq/UW+GeWD NB8DthfRp021gOwDlIj3c7YkqwKBom9TTiW+SDgXllKZvYA5gq7lUlDdbC/1zajstzzI 41O4je9RaBDhga4wjXVo9wgmActw4niD1jYtQYgRLsMs+7Xws0lZccIAUVe92INWgpZ4 RDk/FXJfX8ogA68riMvSsRmxk+eVcdc298+fceQhOEOVm3DRa2FXP5UZcLB4dAnjBRCc pxCA== X-Gm-Message-State: AOAM531UPFSWDaWjXv6xPmqjks814mFDalVgoqARg0xnU5Tsx9BMDoxt lWz3Iy1JKrRET6d2OQSdVhgFyZkNeKk= X-Received: by 2002:a17:902:8509:: with SMTP id bj9mr27287428plb.151.1591146811617; Tue, 02 Jun 2020 18:13:31 -0700 (PDT) Received: from localhost.localdomain (174-21-143-238.tukw.qwest.net. [174.21.143.238]) by smtp.gmail.com with ESMTPSA id 3sm290067pfe.85.2020.06.02.18.13.30 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 02 Jun 2020 18:13:30 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v7 10/42] target/arm: Implement the ADDG, SUBG instructions Date: Tue, 2 Jun 2020 18:12:45 -0700 Message-Id: <20200603011317.473934-11-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200603011317.473934-1-richard.henderson@linaro.org> References: <20200603011317.473934-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::644; envelope-from=richard.henderson@linaro.org; helo=mail-pl1-x644.google.com X-detected-operating-system: by eggs.gnu.org: No matching host in p0f cache. That's all we know. X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: peter.maydell@linaro.org, qemu-arm@nongnu.org, steplong@quicinc.com Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" Signed-off-by: Richard Henderson --- v2: Shift offset in translate; use extract32. v6: Implement inline for !ATA. --- target/arm/helper-a64.h | 1 + target/arm/internals.h | 9 +++++++++ target/arm/mte_helper.c | 10 ++++++++++ target/arm/translate-a64.c | 36 ++++++++++++++++++++++++++++++++++-- 4 files changed, 54 insertions(+), 2 deletions(-) -- 2.25.1 Reviewed-by: Peter Maydell diff --git a/target/arm/helper-a64.h b/target/arm/helper-a64.h index 587ccbe42f..6c116481e8 100644 --- a/target/arm/helper-a64.h +++ b/target/arm/helper-a64.h @@ -105,3 +105,4 @@ DEF_HELPER_FLAGS_2(xpaci, TCG_CALL_NO_RWG_SE, i64, env, i64) DEF_HELPER_FLAGS_2(xpacd, TCG_CALL_NO_RWG_SE, i64, env, i64) DEF_HELPER_FLAGS_3(irg, TCG_CALL_NO_RWG, i64, env, i64, i64) +DEF_HELPER_FLAGS_4(addsubg, TCG_CALL_NO_RWG_SE, i64, env, i64, s32, i32) diff --git a/target/arm/internals.h b/target/arm/internals.h index ae611a6ff5..5c69d4e5a5 100644 --- a/target/arm/internals.h +++ b/target/arm/internals.h @@ -1261,6 +1261,15 @@ void arm_log_exception(int idx); */ #define GMID_EL1_BS 6 +/* We associate one allocation tag per 16 bytes, the minimum. */ +#define LOG2_TAG_GRANULE 4 +#define TAG_GRANULE (1 << LOG2_TAG_GRANULE) + +static inline int allocation_tag_from_addr(uint64_t ptr) +{ + return extract64(ptr, 56, 4); +} + static inline uint64_t address_with_allocation_tag(uint64_t ptr, int rtag) { return deposit64(ptr, 56, 4, rtag); diff --git a/target/arm/mte_helper.c b/target/arm/mte_helper.c index 539a04de84..9ab9ed749d 100644 --- a/target/arm/mte_helper.c +++ b/target/arm/mte_helper.c @@ -70,3 +70,13 @@ uint64_t HELPER(irg)(CPUARMState *env, uint64_t rn, uint64_t rm) return address_with_allocation_tag(rn, rtag); } + +uint64_t HELPER(addsubg)(CPUARMState *env, uint64_t ptr, + int32_t offset, uint32_t tag_offset) +{ + int start_tag = allocation_tag_from_addr(ptr); + uint16_t exclude = extract32(env->cp15.gcr_el1, 0, 16); + int rtag = choose_nonexcluded_tag(start_tag, tag_offset, exclude); + + return address_with_allocation_tag(ptr + offset, rtag); +} diff --git a/target/arm/translate-a64.c b/target/arm/translate-a64.c index 2481561925..a18d71ad98 100644 --- a/target/arm/translate-a64.c +++ b/target/arm/translate-a64.c @@ -3753,17 +3753,20 @@ static void disas_pc_rel_adr(DisasContext *s, uint32_t insn) * sf: 0 -> 32bit, 1 -> 64bit * op: 0 -> add , 1 -> sub * S: 1 -> set flags - * shift: 00 -> LSL imm by 0, 01 -> LSL imm by 12 + * shift: 00 -> LSL imm by 0, + * 01 -> LSL imm by 12 + * 10 -> ADDG, SUBG */ static void disas_add_sub_imm(DisasContext *s, uint32_t insn) { int rd = extract32(insn, 0, 5); int rn = extract32(insn, 5, 5); - uint64_t imm = extract32(insn, 10, 12); + int imm = extract32(insn, 10, 12); int shift = extract32(insn, 22, 2); bool setflags = extract32(insn, 29, 1); bool sub_op = extract32(insn, 30, 1); bool is_64bit = extract32(insn, 31, 1); + bool is_tag = false; TCGv_i64 tcg_rn = cpu_reg_sp(s, rn); TCGv_i64 tcg_rd = setflags ? cpu_reg(s, rd) : cpu_reg_sp(s, rd); @@ -3775,11 +3778,40 @@ static void disas_add_sub_imm(DisasContext *s, uint32_t insn) case 0x1: imm <<= 12; break; + case 0x2: + /* ADDG, SUBG */ + if (!is_64bit || setflags || (imm & 0x30) || + !dc_isar_feature(aa64_mte_insn_reg, s)) { + goto do_unallocated; + } + is_tag = true; + break; default: + do_unallocated: unallocated_encoding(s); return; } + if (is_tag) { + imm = (imm >> 6) << LOG2_TAG_GRANULE; + if (sub_op) { + imm = -imm; + } + + if (s->ata) { + TCGv_i32 tag_offset = tcg_const_i32(imm & 15); + TCGv_i32 offset = tcg_const_i32(imm); + + gen_helper_addsubg(tcg_rd, cpu_env, tcg_rn, offset, tag_offset); + tcg_temp_free_i32(tag_offset); + tcg_temp_free_i32(offset); + } else { + tcg_gen_addi_i64(tcg_rd, tcg_rn, imm); + gen_address_with_allocation_tag0(tcg_rd, tcg_rd); + } + return; + } + tcg_result = tcg_temp_new_i64(); if (!setflags) { if (sub_op) { From patchwork Wed Jun 3 01:12:46 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 187414 Delivered-To: patch@linaro.org Received: by 2002:a92:cf06:0:0:0:0:0 with SMTP id c6csp73523ilo; Tue, 2 Jun 2020 18:20:15 -0700 (PDT) X-Google-Smtp-Source: ABdhPJwolEpLxcbBeD/2SM0Iwz2+QV1EipOBnL2uIxt2VgHYFGAMcIySVsNXI7ujE1eFMyI93fMJ X-Received: by 2002:a25:3413:: with SMTP id b19mr49989266yba.473.1591147215409; Tue, 02 Jun 2020 18:20:15 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1591147215; cv=none; d=google.com; s=arc-20160816; b=DEQ0GwfiPnQ9icIvNtilVlTl2qGejtp4YHXPZB36Jx0dN/iAnj1Jtz6mAY8xbDvgLD DiJiN1Y4jhQWGvC6I1UIzrbzGB7TG5XepTFbklcHDz4tZPQ4vab+mZe08iXq4nzehki7 KRsP0QDinJABBUqlj13O6OtgK8Fk7TAl/sG41Fzz8s0eJdh9f8Kqqzn5UpOMq0A7ANIJ 7oOvhNGE/HNmGBkcjTyajFSUjt8wCrum3Acy2WIiT4MQfb/XhZ7XEs4BLe700KGk8uHn Y5lXVnt+lelT76uwrzImsMp2amnYV4mWreAcmE7xHZyXy3DvyCsknfaqY6ZAEFtBFmeB CObQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=e60Bq5SlAYtW55DZqRQCRGGXcLnTZe8NHr+quQk/wbA=; b=IDjP+52Nqn/oVB8qVu6PvZWePCLylk4MR4vR+wQLsfVxybK3pOLolqSAc/YuVO09bA yWPgCNVRTvVRPp2DgwpDQ3oFoVqA6nPiwoLx9iwOliG4k5SCPoYfuAirt3EjtneOXwS0 9uqmtAP7wbo7utk7WzUXHmccf78J3PIYXxVXgjfbVJN56QaBRVV1t9WFHV5xH8iUsl6r P+CV9VGzaZTLiXiqydrXnRHvnElH4Ddr2BJaJrAHAAoQ0F/QSmCTfGYikwks+obkq1bc WSlK1fZ4OeNJUgUqSPv3v/pj+Al5D+4CHClIPN5/Z4xbxdHSr9zkIzgpM8/zPR/vLfTL M3Dg== ARC-Authentication-Results: i=1; mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=NtTPwYY9; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id 9si328379ybo.254.2020.06.02.18.20.15 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 02 Jun 2020 18:20:15 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=NtTPwYY9; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:48814 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jgI4k-00087q-RH for patch@linaro.org; Tue, 02 Jun 2020 21:20:14 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:48646) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jgHyM-0003qb-2J for qemu-devel@nongnu.org; Tue, 02 Jun 2020 21:13:39 -0400 Received: from mail-pl1-x641.google.com ([2607:f8b0:4864:20::641]:42494) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1jgHyJ-0003YW-QE for qemu-devel@nongnu.org; Tue, 02 Jun 2020 21:13:36 -0400 Received: by mail-pl1-x641.google.com with SMTP id x11so194106plv.9 for ; Tue, 02 Jun 2020 18:13:34 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=e60Bq5SlAYtW55DZqRQCRGGXcLnTZe8NHr+quQk/wbA=; b=NtTPwYY9PU7sn13lSWl2GMDyhBoH9IxK9qoGwFkNqo4bECzelUhm29RunktbGUOOHm AftSaMCpkEW2V7h9I0hct+lLkKdxu+OvHcTUXhYqf0q3xXNzR4CJlz4Pw0a1FKqdZ+H3 6xoRoleY5oPiG3/qh0KUe5u/jaTDNFOWxrxhctPlmbINs3Dc6h/LM02IP9SYOX1DcOBa 0cqiDPLxUXcVdGhhHawA5cMMVfLfu2RyrzrKNbSMjZWqrxsitWfo8B8mTgXI86c1XKce JMYgc6wi1EgFXBDfEZrTtPEyetq8toYyxIacCaXzfHWAvgPWno3GAZoXcdj9MJRyNOsh YBew== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=e60Bq5SlAYtW55DZqRQCRGGXcLnTZe8NHr+quQk/wbA=; b=DAZWH02HhgjXEBzkYrlp96KZGIEAHkvA2W6XZ+wnyLQZFN9FvbuI84QKzy0lErmYz6 Roufs6d46eWYg3oHbyE9UxQA2XHB/07AiFqMrveCIFDvzjbrfMGnvldbhoAy0XYQQPas imacr4BWruFRLPEvMf2e+5JemSAnXr0Mg+nTXLOZlIG7vAXqxKBku3moWoISKj0YgB0w Bl4KIJmYj6dcxVINh73XW1slrcvmF+dAXLv3CUKiMNZ4A0rVFEQ8QDaeV+8M9X/+w/4p co+wcXQblNwat7MUbcMblDzdp/1o8m/9NFi5TfK/d6l/ghKEkSGPnoJaBZ8WWmNcL01N nUOA== X-Gm-Message-State: AOAM5303wAm/aGi/olhsS37d2UDQ+Qg+uw0uQtZC+EYn771oQ26TVdTq sQFAN8PURa6goAXrZdjiZrD1qFyKT5E= X-Received: by 2002:a17:902:7783:: with SMTP id o3mr26908788pll.286.1591146812885; Tue, 02 Jun 2020 18:13:32 -0700 (PDT) Received: from localhost.localdomain (174-21-143-238.tukw.qwest.net. [174.21.143.238]) by smtp.gmail.com with ESMTPSA id 3sm290067pfe.85.2020.06.02.18.13.31 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 02 Jun 2020 18:13:32 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v7 11/42] target/arm: Implement the GMI instruction Date: Tue, 2 Jun 2020 18:12:46 -0700 Message-Id: <20200603011317.473934-12-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200603011317.473934-1-richard.henderson@linaro.org> References: <20200603011317.473934-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::641; envelope-from=richard.henderson@linaro.org; helo=mail-pl1-x641.google.com X-detected-operating-system: by eggs.gnu.org: No matching host in p0f cache. That's all we know. X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: peter.maydell@linaro.org, qemu-arm@nongnu.org, steplong@quicinc.com Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" Signed-off-by: Richard Henderson --- v6: Inline the operation. --- target/arm/translate-a64.c | 15 +++++++++++++++ 1 file changed, 15 insertions(+) -- 2.25.1 Reviewed-by: Peter Maydell diff --git a/target/arm/translate-a64.c b/target/arm/translate-a64.c index a18d71ad98..7d0b7d5b58 100644 --- a/target/arm/translate-a64.c +++ b/target/arm/translate-a64.c @@ -5325,6 +5325,21 @@ static void disas_data_proc_2src(DisasContext *s, uint32_t insn) cpu_reg_sp(s, rn)); } break; + case 5: /* GMI */ + if (sf == 0 || !dc_isar_feature(aa64_mte_insn_reg, s)) { + goto do_unallocated; + } else { + TCGv_i64 t1 = tcg_const_i64(1); + TCGv_i64 t2 = tcg_temp_new_i64(); + + tcg_gen_extract_i64(t2, cpu_reg_sp(s, rn), 56, 4); + tcg_gen_shl_i64(t1, t1, t2); + tcg_gen_or_i64(cpu_reg(s, rd), cpu_reg(s, rm), t1); + + tcg_temp_free_i64(t1); + tcg_temp_free_i64(t2); + } + break; case 8: /* LSLV */ handle_shift_reg(s, A64_SHIFT_TYPE_LSL, sf, rm, rn, rd); break; From patchwork Wed Jun 3 01:12:47 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 187419 Delivered-To: patch@linaro.org Received: by 2002:a92:cf06:0:0:0:0:0 with SMTP id c6csp74647ilo; Tue, 2 Jun 2020 18:22:08 -0700 (PDT) X-Google-Smtp-Source: ABdhPJw0cL4Bs9pLe6IA/H4tiNTiEIhzmP+gAN+/WrEO281MW1iO1yM04UpZgP6P5FvTC4L/4021 X-Received: by 2002:a25:804b:: with SMTP id a11mr17736694ybn.84.1591147328595; Tue, 02 Jun 2020 18:22:08 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1591147328; cv=none; d=google.com; s=arc-20160816; b=wd90HbLn0/UkS4X64dZC1U9v4QVwDI4fYKyZEJ0QBwGK4qoQ2QHsvyqOOJI3N2hwtV hyvyWKc+GDE+XN6LwBtM1l9WHEGXcxEpJhKsLP7DnE6DlceSQuuABR5EEgfJmZ36T5NV EdhKuyrPC+iGBYgIJhYScqwdEjcLCPt+ML+symiTDPYORCGZYUdkBuGDaC3xWafX2Q7s /xpy0sO36+HzY45XtNJ7lq2w4NV+ZpAFFzaZszL9D1Q0S08XRvsI1wCr1n0EQIY+jZKH GdjMsWxC6WzPsMRSiH2gzZdnwUaq9qIxTsKYe4IM0qb10hjPhLpz+hqOE1jYFx/h0snr 2xRw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=TDw09kqIkbskRr4/kNgVUuxCkd+nzaukf7UqdETf1U0=; b=s9fhkgYhWeAPKMFjnuCtfL02wR3VelW3h7tBRwB3OPKGcvZV3QrvMbR5wEEq9klo9G 8C9H2WrlYJLAxHY9KzOFWSOBDAxR9bHpoUbv5Hmv/EWlgOmkBUGm69u7YNsDRmrQjiTP mlZxaNnCyy1M6ktqd7HFOoZyJM+IjGSAi5ny1ae8sTTYrsTTPHko8/klbZAzBeikwF2A MtopK7L7mzMe6rfY1sLChDFQvX66VLSTiLA8dlXjxdUm3rfpH6LslecAAtsPDjuSPOy0 hbeFb39RjuRbprvb4ThlMK0xGiDT95NEGpKJRKUbKkrCIswXWLoz/ev23FIbGQtFiWnm BIXw== ARC-Authentication-Results: i=1; mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=yUo6adD8; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id l4si429449ybt.230.2020.06.02.18.22.08 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 02 Jun 2020 18:22:08 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=yUo6adD8; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:57392 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jgI6a-0003Fe-2w for patch@linaro.org; Tue, 02 Jun 2020 21:22:08 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:48668) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jgHyN-0003rc-QN for qemu-devel@nongnu.org; Tue, 02 Jun 2020 21:13:41 -0400 Received: from mail-pl1-x642.google.com ([2607:f8b0:4864:20::642]:33270) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1jgHyJ-0003Yk-RF for qemu-devel@nongnu.org; Tue, 02 Jun 2020 21:13:39 -0400 Received: by mail-pl1-x642.google.com with SMTP id t7so209382plr.0 for ; Tue, 02 Jun 2020 18:13:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=TDw09kqIkbskRr4/kNgVUuxCkd+nzaukf7UqdETf1U0=; b=yUo6adD8xnB39ooYZVKvAMUokWhucdRFGYuFbSDUjdog0eTz3gh6UKycWetGt9Q+fB CgLP6dzFgmkT/9YSLPYp1iqba8ULo7ZkZNDC/NBIxXJFKw/Q7cBKQq84btLmQw4qX42p vZsQH52cJjxoP8w2wNaLAzr7jD788JtY+nkwld5Hyjv+cLpz/5PDleNX0LBVjDpzfiFZ ZRXk5w6qBa2KWaAE5hd6qCTKihBIX73UVJfFBsnW7k//6EAyVDfU0zwjevbgKDqzaNqT AyeIhnzyjMp/hNFOVF+dyUou6/sG0RNFNXY5BJ4AweqwQSNPb8+0iqGAU9VM1bk5mZFl kbYw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=TDw09kqIkbskRr4/kNgVUuxCkd+nzaukf7UqdETf1U0=; b=kV7fDp7YBYjw5ZiynsRSt083j4o7PrEKmCksXWUpOSNJjI/q2uFCWqAKC+A0J95SQ4 CpIcjNlM9bbR9WmhXYnRO0HNpcEY2c0TlDjch70CY2r9wJ6gmuMSpwJe+S4kkQa8I7/4 sDKRr3z3WhntaikNmhIsfxTPp485WpaWuagWqpfAhLSHHiBCyFoFcudJCdUiaVaz5U/e rAayjcZQHjpSG2C/m4N8THs7x/PzOhgC8Jv+/IXq/QPrBGxW2dGV73MAlgcDvnIGgi/9 oDj3x6CGDhIzX8+7gpsoB2VLinJgfE+X7+MYe0FTI0L5/6eQamxGU1graCAyqL9IsjmR Wqgw== X-Gm-Message-State: AOAM533ExE2trNdVylyWP5q/P4pygCRyilCN8SeOad+JHQb57PI399Ww Uodv0scrX+bhq9hTdvGC4+mpWNs/Zrw= X-Received: by 2002:a17:902:aa92:: with SMTP id d18mr27914389plr.127.1591146814331; Tue, 02 Jun 2020 18:13:34 -0700 (PDT) Received: from localhost.localdomain (174-21-143-238.tukw.qwest.net. [174.21.143.238]) by smtp.gmail.com with ESMTPSA id 3sm290067pfe.85.2020.06.02.18.13.33 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 02 Jun 2020 18:13:33 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v7 12/42] target/arm: Implement the SUBP instruction Date: Tue, 2 Jun 2020 18:12:47 -0700 Message-Id: <20200603011317.473934-13-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200603011317.473934-1-richard.henderson@linaro.org> References: <20200603011317.473934-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::642; envelope-from=richard.henderson@linaro.org; helo=mail-pl1-x642.google.com X-detected-operating-system: by eggs.gnu.org: No matching host in p0f cache. That's all we know. X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: peter.maydell@linaro.org, qemu-arm@nongnu.org, steplong@quicinc.com Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" Reviewed-by: Peter Maydell Signed-off-by: Richard Henderson --- v2: Fix extraction length. --- target/arm/translate-a64.c | 24 ++++++++++++++++++++++-- 1 file changed, 22 insertions(+), 2 deletions(-) -- 2.25.1 diff --git a/target/arm/translate-a64.c b/target/arm/translate-a64.c index 7d0b7d5b58..0ee2ef403e 100644 --- a/target/arm/translate-a64.c +++ b/target/arm/translate-a64.c @@ -5294,19 +5294,39 @@ static void handle_crc32(DisasContext *s, */ static void disas_data_proc_2src(DisasContext *s, uint32_t insn) { - unsigned int sf, rm, opcode, rn, rd; + unsigned int sf, rm, opcode, rn, rd, setflag; sf = extract32(insn, 31, 1); + setflag = extract32(insn, 29, 1); rm = extract32(insn, 16, 5); opcode = extract32(insn, 10, 6); rn = extract32(insn, 5, 5); rd = extract32(insn, 0, 5); - if (extract32(insn, 29, 1)) { + if (setflag && opcode != 0) { unallocated_encoding(s); return; } switch (opcode) { + case 0: /* SUBP(S) */ + if (sf == 0 || !dc_isar_feature(aa64_mte_insn_reg, s)) { + goto do_unallocated; + } else { + TCGv_i64 tcg_n, tcg_m, tcg_d; + + tcg_n = read_cpu_reg_sp(s, rn, true); + tcg_m = read_cpu_reg_sp(s, rm, true); + tcg_gen_sextract_i64(tcg_n, tcg_n, 0, 56); + tcg_gen_sextract_i64(tcg_m, tcg_m, 0, 56); + tcg_d = cpu_reg(s, rd); + + if (setflag) { + gen_sub_CC(true, tcg_d, tcg_n, tcg_m); + } else { + tcg_gen_sub_i64(tcg_d, tcg_n, tcg_m); + } + } + break; case 2: /* UDIV */ handle_div(s, false, sf, rm, rn, rd); break; From patchwork Wed Jun 3 01:12:48 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 187417 Delivered-To: patch@linaro.org Received: by 2002:a92:cf06:0:0:0:0:0 with SMTP id c6csp74302ilo; Tue, 2 Jun 2020 18:21:33 -0700 (PDT) X-Google-Smtp-Source: ABdhPJwJHn3v90Q7hUHyiRTMgzgU/IIAC9w/330fpxZhNF7pgCM/O6xDtj0TIdl3484XcwFBvVmV X-Received: by 2002:a5b:949:: with SMTP id x9mr10387202ybq.222.1591147293727; Tue, 02 Jun 2020 18:21:33 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1591147293; cv=none; d=google.com; s=arc-20160816; b=qxnixH3nns7aY0oy/5/IalYtDzG1i8cWV3a5WxTSqqFpn9PbBkbmLccteJb113WhFe hnaiDhODwtMgS1BlqfkImrgv2zXYz/Vr2v6MqdhfGfKLcVlTJjOrGQ9/1SeAhGCWztz8 MlKpYCz+UluL3JdvcYfdTz8UFTuGpnuvQW40g/3fDNQ0hXPwDR8dzsh48SWV1Uf/785Z 11I4eyEXOIMjE7qMOsWhNXa6U6WBpy0ekWCh0cFPiKGXMfPR8a2Xmig8nzObS1KSv+7t d86+S1wLSw3HBjmECUpQUz0q37p8LkQeCXvYJ26g/IEjatCEwfQI7WQGctIJKnh1bpiK ocbA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=0EIkcVZM7CTGPx7yRFPLzi2q/BnEPgFyUzrQe3RUVmw=; b=vSI8PRgD5eWEtmkBTDG3/yFofn1AqPXe3IteUYov4/e7NS29nr/7E3VVsZi8LlfZbe rZyPuyvqCiQkC6oNW64utZe5MOHOAi9zujG10PiN0O2KgMao0B/iNokp4Knczb0K/m2a zb5abNt67d6TkN5No4Fz0SWaT9S4wG/ELvDd3XObxYPpsNYSoT1wn4w2imPOzoF1d9d5 00SWxhtjsLb5SZigeDWoqw3Ko67EYgaxBO39R4OPFSs+kWPXfuniF8EtYA32+y5ds2UC 7rE9bxvEYBhrr0aFOcVgwsxiRnDQdkGry1A8JtuwQsMtHAapzD5+XKWxMn01UnVPpfxH Q74g== ARC-Authentication-Results: i=1; mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=A3TJV5nC; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id h13si278429ybq.411.2020.06.02.18.21.33 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 02 Jun 2020 18:21:33 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=A3TJV5nC; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:55832 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jgI61-0002ca-4z for patch@linaro.org; Tue, 02 Jun 2020 21:21:33 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:48712) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jgHyT-0003yC-9e for qemu-devel@nongnu.org; Tue, 02 Jun 2020 21:13:45 -0400 Received: from mail-pl1-x642.google.com ([2607:f8b0:4864:20::642]:45833) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1jgHyL-0003Yx-Pt for qemu-devel@nongnu.org; Tue, 02 Jun 2020 21:13:43 -0400 Received: by mail-pl1-x642.google.com with SMTP id y11so187072plt.12 for ; Tue, 02 Jun 2020 18:13:37 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=0EIkcVZM7CTGPx7yRFPLzi2q/BnEPgFyUzrQe3RUVmw=; b=A3TJV5nCnr3oYovFU1ADFd+5yfFGhL0YsAB6KWudClLp1wvvmNicYYxE+o0VmSob5o p138YpdPOZiQNyBhJmc2bHHEra6CcwLLkoatJTvmxsPrgGSSvEAF+e1+6S886KwhNE9q DOSTP2M15CU1MV1HQMlQ4H9byezhmyDDVF6j6FHTcFkBsx+AIvklRPWXzbStPUSYl116 vacfPn1/3C3TUSMtjVRqiATU/ah4CEfQA/3gnBiXw7lLJwoW8+9ldG9hbcDDQGl27Vmu SBfWVCdk6F4oUAXCg0bSLVlM2TKvyudy1T/vvlfpgnwX87pb8GtZY9kwnZp4SzUsQ2iq 6xLg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=0EIkcVZM7CTGPx7yRFPLzi2q/BnEPgFyUzrQe3RUVmw=; b=Pww4HSDBFug91K8TdPhrTiA8WkwGWrZqxZRAxq4xulafm1SPNsScZnqVX32TKMzb9p qzf+JZe0rmSD2f+L47kS7mr4t+Xpwlf9KSoGdod56VWJBKgRrc3NZy66kzFGBrn/Fpsm IDejndqRGN8wVirl/HQZRYmgKb2HErW627bDm92DoM+WjVgxiYym0IuKEuXzdo9n3aC3 XX1zZ2s5rrwQbXmLvgQmeUsSs4gSJENJKDmPvFx+TTEYNAf4pDRAQjkqCfzip30hp9+G IIEpSqIqfTdiGdoit7vosVq0Rok5WV1cD/tgzOlPMTN2drNIbkcP0ZdTlN2HQjysi06f ZGmA== X-Gm-Message-State: AOAM533JIAcsULGHFEZav9g2uBwwHkhZza19h73AUiWg0SlF4zABtUTf 8rqMuZ6wRwX37lnUWfDP52Lzi+1QDYU= X-Received: by 2002:a17:902:8218:: with SMTP id x24mr27620797pln.150.1591146815640; Tue, 02 Jun 2020 18:13:35 -0700 (PDT) Received: from localhost.localdomain (174-21-143-238.tukw.qwest.net. [174.21.143.238]) by smtp.gmail.com with ESMTPSA id 3sm290067pfe.85.2020.06.02.18.13.34 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 02 Jun 2020 18:13:34 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v7 13/42] target/arm: Define arm_cpu_do_unaligned_access for user-only Date: Tue, 2 Jun 2020 18:12:48 -0700 Message-Id: <20200603011317.473934-14-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200603011317.473934-1-richard.henderson@linaro.org> References: <20200603011317.473934-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::642; envelope-from=richard.henderson@linaro.org; helo=mail-pl1-x642.google.com X-detected-operating-system: by eggs.gnu.org: No matching host in p0f cache. That's all we know. X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: peter.maydell@linaro.org, qemu-arm@nongnu.org, steplong@quicinc.com Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" We need this to raise unaligned exceptions from user mode. Signed-off-by: Richard Henderson --- v6: Use EXCP_UNALIGNED for user-only and update cpu_loop.c. --- linux-user/aarch64/cpu_loop.c | 7 ++++++ linux-user/arm/cpu_loop.c | 7 ++++++ target/arm/cpu.c | 2 +- target/arm/tlb_helper.c | 41 ++++++++++++++++++++++------------- 4 files changed, 41 insertions(+), 16 deletions(-) -- 2.25.1 Reviewed-by: Peter Maydell diff --git a/linux-user/aarch64/cpu_loop.c b/linux-user/aarch64/cpu_loop.c index bbe9fefca8..3cca637bb9 100644 --- a/linux-user/aarch64/cpu_loop.c +++ b/linux-user/aarch64/cpu_loop.c @@ -121,6 +121,13 @@ void cpu_loop(CPUARMState *env) info._sifields._sigfault._addr = env->exception.vaddress; queue_signal(env, info.si_signo, QEMU_SI_FAULT, &info); break; + case EXCP_UNALIGNED: + info.si_signo = TARGET_SIGBUS; + info.si_errno = 0; + info.si_code = TARGET_BUS_ADRALN; + info._sifields._sigfault._addr = env->exception.vaddress; + queue_signal(env, info.si_signo, QEMU_SI_FAULT, &info); + break; case EXCP_DEBUG: case EXCP_BKPT: info.si_signo = TARGET_SIGTRAP; diff --git a/linux-user/arm/cpu_loop.c b/linux-user/arm/cpu_loop.c index 13629ee1f6..0d568d2d69 100644 --- a/linux-user/arm/cpu_loop.c +++ b/linux-user/arm/cpu_loop.c @@ -411,6 +411,13 @@ void cpu_loop(CPUARMState *env) queue_signal(env, info.si_signo, QEMU_SI_FAULT, &info); } break; + case EXCP_UNALIGNED: + info.si_signo = TARGET_SIGBUS; + info.si_errno = 0; + info.si_code = TARGET_BUS_ADRALN; + info._sifields._sigfault._addr = env->exception.vaddress; + queue_signal(env, info.si_signo, QEMU_SI_FAULT, &info); + break; case EXCP_DEBUG: case EXCP_BKPT: excp_debug: diff --git a/target/arm/cpu.c b/target/arm/cpu.c index 32bec156f2..0f1a46f531 100644 --- a/target/arm/cpu.c +++ b/target/arm/cpu.c @@ -2166,8 +2166,8 @@ static void arm_cpu_class_init(ObjectClass *oc, void *data) cc->tlb_fill = arm_cpu_tlb_fill; cc->debug_excp_handler = arm_debug_excp_handler; cc->debug_check_watchpoint = arm_debug_check_watchpoint; -#if !defined(CONFIG_USER_ONLY) cc->do_unaligned_access = arm_cpu_do_unaligned_access; +#if !defined(CONFIG_USER_ONLY) cc->do_transaction_failed = arm_cpu_do_transaction_failed; cc->adjust_watchpoint_address = arm_adjust_watchpoint_address; #endif /* CONFIG_TCG && !CONFIG_USER_ONLY */ diff --git a/target/arm/tlb_helper.c b/target/arm/tlb_helper.c index 7388494a55..d4e6d37f4f 100644 --- a/target/arm/tlb_helper.c +++ b/target/arm/tlb_helper.c @@ -107,21 +107,6 @@ static void QEMU_NORETURN arm_deliver_fault(ARMCPU *cpu, vaddr addr, raise_exception(env, exc, syn, target_el); } -/* Raise a data fault alignment exception for the specified virtual address */ -void arm_cpu_do_unaligned_access(CPUState *cs, vaddr vaddr, - MMUAccessType access_type, - int mmu_idx, uintptr_t retaddr) -{ - ARMCPU *cpu = ARM_CPU(cs); - ARMMMUFaultInfo fi = {}; - - /* now we have a real cpu fault */ - cpu_restore_state(cs, retaddr, true); - - fi.type = ARMFault_Alignment; - arm_deliver_fault(cpu, vaddr, access_type, mmu_idx, &fi); -} - /* * arm_cpu_do_transaction_failed: handle a memory system error response * (eg "no device/memory present at address") by raising an external abort @@ -198,3 +183,29 @@ bool arm_cpu_tlb_fill(CPUState *cs, vaddr address, int size, } #endif } + +/* Raise a data fault alignment exception for the specified virtual address */ +void arm_cpu_do_unaligned_access(CPUState *cs, vaddr vaddr, + MMUAccessType access_type, + int mmu_idx, uintptr_t retaddr) +{ + ARMCPU *cpu = ARM_CPU(cs); + +#ifdef CONFIG_USER_ONLY + cpu->env.exception.vaddress = vaddr; + /* + * For HW, this is EXCP_DATA_ABORT with a proper syndrome. + * Make it easier for ourselves in linux-user/arm/cpu_loop.c. + */ + cs->exception_index = EXCP_UNALIGNED; + cpu_loop_exit_restore(cs, retaddr); +#else + ARMMMUFaultInfo fi = {}; + + /* now we have a real cpu fault */ + cpu_restore_state(cs, retaddr, true); + + fi.type = ARMFault_Alignment; + arm_deliver_fault(cpu, vaddr, access_type, mmu_idx, &fi); +#endif +} From patchwork Wed Jun 3 01:12:49 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 187416 Delivered-To: patch@linaro.org Received: by 2002:a92:cf06:0:0:0:0:0 with SMTP id c6csp73820ilo; Tue, 2 Jun 2020 18:20:45 -0700 (PDT) X-Google-Smtp-Source: ABdhPJzIImPhk2UO+obbIhvFUYL9LxkJ/FRzUn8NVhvmoQfnL64Fz2tq6Ufd+UyuQr0YMKg2p6mo X-Received: by 2002:a25:a063:: with SMTP id x90mr331025ybh.191.1591147244940; Tue, 02 Jun 2020 18:20:44 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1591147244; cv=none; d=google.com; s=arc-20160816; b=Vw/k/HHYV2zG2T1U3MfFBr3o5je57icMw9f2x9wZJfyxWoY5IHwubdq3rJ2Q1DOjEc 0tTbulgC5cQHIIc6V+u9JDmLxtFDTtjQSwmk/embM0z0HRtehIo/Q+nWYWJz7erGjbDi xJJDi0BfSHfo8wi/fOd9vucxWN+8zXg1Vz23kMU1qQyN4Na7uajhy49pZdFFVMUy9t3+ ihA7VaYX3vW1GgWh4RjFyrGbRn8Ei/Wz+hFGSMieF3rb8957+/dXaze8QAzNkINW/gJS h5E65SVUIIBqhlrbZid7mW6rAil0StQjT4xkr1gLHpFWAUtUb6u+hIKZ8zOjMlGU3nHu KUbg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=61m39+FMAykppXrhX2BpqO13HC/u4DFx0okzBeTOBpE=; b=z0W9cYvD5mDnyQoUSseYu0DYmq/ThfLc4Ma7gIUop0h8GPjuhU+Dcj1qXxd6x1lQoe r3rcjYB1latbiOcJg10O4KqWLPT8Xm0WzF0wII7mKGhpA8uR01vGFgQYVNYro6wrC7xd /ic8E7AByZwd3Rm+BliepY827zXjRAnwl7gC23XDXJ6qdDZyQXUdCX+H6FWxBGPnjNfg KcWWmY5kzuGxSzDQu40PtL1kjq5eCowU1t7avqEdPo9anR7AIsjWuGcCk/WFnW7Dd1S6 030DNMtUd/SxQYJyxQoNyJkYXUVzTou/mj7T7DQX8jmWq0B1NtkkvKTvxRBFhyQpPsKG +TzA== ARC-Authentication-Results: i=1; mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=a15OZsXw; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id v16si248235ybe.324.2020.06.02.18.20.44 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 02 Jun 2020 18:20:44 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=a15OZsXw; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:47738 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jgI5E-0007i0-Cx for patch@linaro.org; Tue, 02 Jun 2020 21:20:44 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:48760) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jgHyX-00045i-7k for qemu-devel@nongnu.org; Tue, 02 Jun 2020 21:13:49 -0400 Received: from mail-pl1-x644.google.com ([2607:f8b0:4864:20::644]:43511) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1jgHyN-0003Z8-GV for qemu-devel@nongnu.org; Tue, 02 Jun 2020 21:13:48 -0400 Received: by mail-pl1-x644.google.com with SMTP id g12so191223pll.10 for ; Tue, 02 Jun 2020 18:13:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=61m39+FMAykppXrhX2BpqO13HC/u4DFx0okzBeTOBpE=; b=a15OZsXw7InJ70zYYPo+xB1xlJaZFhubTGye7fFD7myJv5BmMISyjlOfz6soUBYY5k wmB9hwcYGkcLmjuVBxf7HkRemEWKYQw0Y/o0dvcXPQ8dfsFUnQgPgZJC3NuRqFGD3TOj 4hHsIOAkIoTxVrtxeHzwG9Y+iQLgWc9nQGzP3jux7L2/a9CUazwXmAVBs7VxZdbdu5+Q iiGY5vkKM9vy7xkoeMg0E27VKEIomKfhy69I1k3HrO6WXBBdCx1uPYbzxXNSQgHEVYBX UtgfL/+fMecWXnYtmndJYeeEeuPLlNMCsmxN9BT5IxWGZ/enWJAhHN8oFDWr6WDsN7cF 2sow== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=61m39+FMAykppXrhX2BpqO13HC/u4DFx0okzBeTOBpE=; b=g4rj2GW4rYlXjX/eESkucmKPZuHnbMRoCNeG0o1zitcw8haTXKhjeHPDbzqUGv3ko+ LevSZIZHRu+MBCnQA+KhRSMX4pPPP4JQzeFfd4xY3dR/ipChT9lNrWhLL552O60ykmBW Rb4jEJeQWmoyalVlaiJxCcMVxEYElCSH1LhvPQyvR/DkF5khgCaoTmXJuJu+FRDekY6E ekDQIzmUQDUEUS+jvGRpLcxx3i8BrTeFWt+pKFcNgCzPbhNBI5XZ5e3HrwqUqxmfMiNI PfzmAt3ZDfeuAif2AeFZsMOeQJO5Zaj4+iwHwSc2iy6y3+bAuDpQ4ojmJoo3D4XyxDK3 NI+A== X-Gm-Message-State: AOAM530K9ljhIXiI8na7qbOe1MHuBu/O1gP2Q4j6Zd9DQj7Y1StbknhR ANI5fDfaL709UJOlO9TkgbrlSejeB0U= X-Received: by 2002:a17:902:b110:: with SMTP id q16mr26887701plr.221.1591146816819; Tue, 02 Jun 2020 18:13:36 -0700 (PDT) Received: from localhost.localdomain (174-21-143-238.tukw.qwest.net. [174.21.143.238]) by smtp.gmail.com with ESMTPSA id 3sm290067pfe.85.2020.06.02.18.13.35 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 02 Jun 2020 18:13:36 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v7 14/42] target/arm: Add helper_probe_access Date: Tue, 2 Jun 2020 18:12:49 -0700 Message-Id: <20200603011317.473934-15-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200603011317.473934-1-richard.henderson@linaro.org> References: <20200603011317.473934-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::644; envelope-from=richard.henderson@linaro.org; helo=mail-pl1-x644.google.com X-detected-operating-system: by eggs.gnu.org: No matching host in p0f cache. That's all we know. X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: peter.maydell@linaro.org, qemu-arm@nongnu.org, steplong@quicinc.com Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" Raise an exception if the given virtual memory is not accessible. Signed-off-by: Richard Henderson --- target/arm/helper.h | 2 ++ target/arm/op_helper.c | 16 ++++++++++++++++ target/arm/translate-a64.c | 13 +++++++++++++ 3 files changed, 31 insertions(+) -- 2.25.1 diff --git a/target/arm/helper.h b/target/arm/helper.h index 49336dc432..4b64d1ebd3 100644 --- a/target/arm/helper.h +++ b/target/arm/helper.h @@ -96,6 +96,8 @@ DEF_HELPER_FLAGS_1(rebuild_hflags_a32_newel, TCG_CALL_NO_RWG, void, env) DEF_HELPER_FLAGS_2(rebuild_hflags_a32, TCG_CALL_NO_RWG, void, env, int) DEF_HELPER_FLAGS_2(rebuild_hflags_a64, TCG_CALL_NO_RWG, void, env, int) +DEF_HELPER_FLAGS_5(probe_access, TCG_CALL_NO_WG, void, env, tl, i32, i32, i32) + DEF_HELPER_1(vfp_get_fpscr, i32, env) DEF_HELPER_2(vfp_set_fpscr, void, env, i32) diff --git a/target/arm/op_helper.c b/target/arm/op_helper.c index eb0de080f1..b1065216b2 100644 --- a/target/arm/op_helper.c +++ b/target/arm/op_helper.c @@ -935,3 +935,19 @@ uint32_t HELPER(ror_cc)(CPUARMState *env, uint32_t x, uint32_t i) return ((uint32_t)x >> shift) | (x << (32 - shift)); } } + +void HELPER(probe_access)(CPUARMState *env, target_ulong ptr, + uint32_t access_type, uint32_t mmu_idx, + uint32_t size) +{ + uint32_t in_page = -((uint32_t)ptr | TARGET_PAGE_SIZE); + uintptr_t ra = GETPC(); + + if (likely(size <= in_page)) { + probe_access(env, ptr, size, access_type, mmu_idx, ra); + } else { + probe_access(env, ptr, in_page, access_type, mmu_idx, ra); + probe_access(env, ptr + in_page, size - in_page, + access_type, mmu_idx, ra); + } +} diff --git a/target/arm/translate-a64.c b/target/arm/translate-a64.c index 0ee2ef403e..b032829194 100644 --- a/target/arm/translate-a64.c +++ b/target/arm/translate-a64.c @@ -232,6 +232,19 @@ static void gen_address_with_allocation_tag0(TCGv_i64 dst, TCGv_i64 src) tcg_gen_andi_i64(dst, src, ~MAKE_64BIT_MASK(56, 4)); } +static void gen_probe_access(DisasContext *s, TCGv_i64 ptr, + MMUAccessType acc, int log2_size) +{ + TCGv_i32 t_acc = tcg_const_i32(acc); + TCGv_i32 t_idx = tcg_const_i32(get_mem_index(s)); + TCGv_i32 t_size = tcg_const_i32(1 << log2_size); + + gen_helper_probe_access(cpu_env, ptr, t_acc, t_idx, t_size); + tcg_temp_free_i32(t_acc); + tcg_temp_free_i32(t_idx); + tcg_temp_free_i32(t_size); +} + typedef struct DisasCompare64 { TCGCond cond; TCGv_i64 value; From patchwork Wed Jun 3 01:12:50 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 187412 Delivered-To: patch@linaro.org Received: by 2002:a92:cf06:0:0:0:0:0 with SMTP id c6csp72661ilo; Tue, 2 Jun 2020 18:18:52 -0700 (PDT) X-Google-Smtp-Source: ABdhPJy5SpH7ZZYO0K3LC6BSHOeyRJVVAL+T4nCpzA9y53feqJFXdyVjimt+AvJjnCJKw34tNCbq X-Received: by 2002:a25:bdd3:: with SMTP id g19mr37300719ybk.5.1591147132412; Tue, 02 Jun 2020 18:18:52 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1591147132; cv=none; d=google.com; s=arc-20160816; b=Ke+rIDQQ0f2m7tjmZxxyEEbPEzn1ce4E7haApT8ZNQ0Ym4kc74ZkQZdD+nAYb9DU+U BaVdewH+v8XwLNUFxwCVHiLcmh6s2jBx/K0XlCFUFYuyh2x8t+7qB6pkhqLTQ0SXf3PD Iqj1prtTTABmfLZfIQ5+5XXMgkTXtKerxU2lJL3JaG7G/00hgpOL4WFVQbKnMCfVuOYw w0zGlLnpANFC2i+ebjezfLAuvvQe3weVsFQH99uky+mkazcQSulm3hCbDpYZchDIsgMj MSFXMmVLy6FYnMTg59PiDOh4SwD/xnQKCWg2oHaa0PG+dsBRpzA1qxRt/ePpJ99qLj05 lF7A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=iD1HCmE/mBT9UdzHtyaiNXiw6VvhW3BvWlEg2J8SAgE=; b=JmiT2W7CS3K39e3dtF+Mz8yeJiheJbVBhPOD2Opce36GzMCvJNmwIcf+sniWSZveVs P1SiVzPLP3FD50dM73kpx9FdZPStmwI47NTv4xClzWxyAAqB6WN43NIK0wBU1u0ud0XY Dxqsbum/PLfRUVYD0LqGc1y+VbgZ61kshG9Lknq0fUZb0GP/LSv+ARuwanGWhi66Zj+y U2vzBGdCAAgRtoa4LvIXUs9Y5TVNBaqeyszX4kZeXEjDZFmE8CgDZKmgefkUkR3Jjhuj vPgldK19GYujzQ8lEGDtuws4tKDiG9NT78nPPTDt4zRvxCSCOsBZ7u49/lsQO83EGGok 3XQg== ARC-Authentication-Results: i=1; mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=s9ZCi8n7; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id w4si293069ybc.406.2020.06.02.18.18.52 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 02 Jun 2020 18:18:52 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=s9ZCi8n7; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:38690 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jgI3P-00049h-SB for patch@linaro.org; Tue, 02 Jun 2020 21:18:51 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:48738) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jgHyV-00041g-RI for qemu-devel@nongnu.org; Tue, 02 Jun 2020 21:13:47 -0400 Received: from mail-pl1-x633.google.com ([2607:f8b0:4864:20::633]:43232) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1jgHyP-0003ZS-By for qemu-devel@nongnu.org; Tue, 02 Jun 2020 21:13:47 -0400 Received: by mail-pl1-x633.google.com with SMTP id g12so191239pll.10 for ; Tue, 02 Jun 2020 18:13:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=iD1HCmE/mBT9UdzHtyaiNXiw6VvhW3BvWlEg2J8SAgE=; b=s9ZCi8n7qawEHTJfszwHzAbPrF2DvVIASVDLbLnXzgdllqPe4dkuFBIlN+e39NCsP7 /LIf5r/0D0GUkKOkgA2BzyKw2UctGsfA6O+eIrPetSZNXFFF8mMDSSB69rC3AP8qtb+3 kOy0AnuisCA1pQAwwr/sakoioyBrPbr79PPbUxOiUuK7fp/jVHSau3q4w6vDFUg/7y45 a8IMjubzJTecYAksyLJvIcdJoYLMKDMgcMl0glsqxynVHKXMjIqn5YzIElLNfRmIaB9W 7K83hKLMNtgdFu/ZiIOZxrsZQMUGbatj+T2JKLF2I5oWetxbpTjInipT6eaFd3ZFBFom hVLg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=iD1HCmE/mBT9UdzHtyaiNXiw6VvhW3BvWlEg2J8SAgE=; b=NtR93CEaesHYCB8dIJlAJQbrSfzjLHkeg4MHLB+swchs5vFV6OpihVJyXUc1c0ICNH HF79VbkDkued6Wt6uciOKUEg5U2yrCZp9Q4quEyUeiWbxrqYoY8BqQS6wsJCQy5U5Q0Z WNeHWWwLnQZs0qF0R34cMxI/DzM7atGALLXOROzo/I8b/lZKPK5gXvgvdviEXysqVyZD 5laBwDo/KfXlZbVnpS9XlVa/LqnFP+sII7uQpz2C3eRDB6ybt0E5k2cacNIl7G/9f5ML giMfqB+8B35Fd/hkg4IHp3LFJxlx9DGBfAGOGN5284rJGz3kTjG2fruvW2L0hIrDg4rm HSnw== X-Gm-Message-State: AOAM530/8keImKFoVd29qvJUEhJ8/yII938AF36E0VsivtWl2wNsvZIX /alpUJeX2kjPGI5fomGG6R8OQ6S78N0= X-Received: by 2002:a17:902:b103:: with SMTP id q3mr21208946plr.265.1591146818066; Tue, 02 Jun 2020 18:13:38 -0700 (PDT) Received: from localhost.localdomain (174-21-143-238.tukw.qwest.net. [174.21.143.238]) by smtp.gmail.com with ESMTPSA id 3sm290067pfe.85.2020.06.02.18.13.36 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 02 Jun 2020 18:13:37 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v7 15/42] target/arm: Implement LDG, STG, ST2G instructions Date: Tue, 2 Jun 2020 18:12:50 -0700 Message-Id: <20200603011317.473934-16-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200603011317.473934-1-richard.henderson@linaro.org> References: <20200603011317.473934-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::633; envelope-from=richard.henderson@linaro.org; helo=mail-pl1-x633.google.com X-detected-operating-system: by eggs.gnu.org: No matching host in p0f cache. That's all we know. X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: peter.maydell@linaro.org, qemu-arm@nongnu.org, steplong@quicinc.com Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" Signed-off-by: Richard Henderson --- v2: Split out allocation_tag_mem. Handle atomicity of stores. v3: Add X[t] input to these insns; require pre-cleaned addresses. v5: Fix !32-byte aligned operation of st2g. v6: Fix op2 extract, stg pre/post-index, stores vs sp, commentary; use pre-computed ata. v7: Fix STZG iteration (stephen long) --- target/arm/helper-a64.h | 7 ++ target/arm/mte_helper.c | 194 +++++++++++++++++++++++++++++++++++++ target/arm/translate-a64.c | 158 +++++++++++++++++++++++++++++- 3 files changed, 354 insertions(+), 5 deletions(-) -- 2.25.1 diff --git a/target/arm/helper-a64.h b/target/arm/helper-a64.h index 6c116481e8..2fa61b86fa 100644 --- a/target/arm/helper-a64.h +++ b/target/arm/helper-a64.h @@ -106,3 +106,10 @@ DEF_HELPER_FLAGS_2(xpacd, TCG_CALL_NO_RWG_SE, i64, env, i64) DEF_HELPER_FLAGS_3(irg, TCG_CALL_NO_RWG, i64, env, i64, i64) DEF_HELPER_FLAGS_4(addsubg, TCG_CALL_NO_RWG_SE, i64, env, i64, s32, i32) +DEF_HELPER_FLAGS_3(ldg, TCG_CALL_NO_WG, i64, env, i64, i64) +DEF_HELPER_FLAGS_3(stg, TCG_CALL_NO_WG, void, env, i64, i64) +DEF_HELPER_FLAGS_3(stg_parallel, TCG_CALL_NO_WG, void, env, i64, i64) +DEF_HELPER_FLAGS_2(stg_stub, TCG_CALL_NO_WG, void, env, i64) +DEF_HELPER_FLAGS_3(st2g, TCG_CALL_NO_WG, void, env, i64, i64) +DEF_HELPER_FLAGS_3(st2g_parallel, TCG_CALL_NO_WG, void, env, i64, i64) +DEF_HELPER_FLAGS_2(st2g_stub, TCG_CALL_NO_WG, void, env, i64) diff --git a/target/arm/mte_helper.c b/target/arm/mte_helper.c index 9ab9ed749d..7ec7930dfc 100644 --- a/target/arm/mte_helper.c +++ b/target/arm/mte_helper.c @@ -44,6 +44,40 @@ static int choose_nonexcluded_tag(int tag, int offset, uint16_t exclude) return tag; } +/** + * allocation_tag_mem: + * @env: the cpu environment + * @ptr_mmu_idx: the addressing regime to use for the virtual address + * @ptr: the virtual address for which to look up tag memory + * @ptr_access: the access to use for the virtual address + * @ptr_size: the number of bytes in the normal memory access + * @tag_access: the access to use for the tag memory + * @tag_size: the number of bytes in the tag memory access + * @ra: the return address for exception handling + * + * Our tag memory is formatted as a sequence of little-endian nibbles. + * That is, the byte at (addr >> (LOG2_TAG_GRANULE + 1)) contains two + * tags, with the tag at [3:0] for the lower addr and the tag at [7:4] + * for the higher addr. + * + * Here, resolve the physical address from the virtual address, and return + * a pointer to the corresponding tag byte. Exit with exception if the + * virtual address is not accessible for @ptr_access. + * + * The @ptr_size and @tag_size values may not have an obvious relation + * due to the alignment of @ptr, and the number of tag checks required. + * + * If there is no tag storage corresponding to @ptr, return NULL. + */ +static uint8_t *allocation_tag_mem(CPUARMState *env, int ptr_mmu_idx, + uint64_t ptr, MMUAccessType ptr_access, + int ptr_size, MMUAccessType tag_access, + int tag_size, uintptr_t ra) +{ + /* Tag storage not implemented. */ + return NULL; +} + uint64_t HELPER(irg)(CPUARMState *env, uint64_t rn, uint64_t rm) { int rtag; @@ -80,3 +114,163 @@ uint64_t HELPER(addsubg)(CPUARMState *env, uint64_t ptr, return address_with_allocation_tag(ptr + offset, rtag); } + +static int load_tag1(uint64_t ptr, uint8_t *mem) +{ + int ofs = extract32(ptr, LOG2_TAG_GRANULE, 1) * 4; + return extract32(*mem, ofs, 4); +} + +uint64_t HELPER(ldg)(CPUARMState *env, uint64_t ptr, uint64_t xt) +{ + int mmu_idx = cpu_mmu_index(env, false); + uint8_t *mem; + int rtag = 0; + + /* Trap if accessing an invalid page. */ + mem = allocation_tag_mem(env, mmu_idx, ptr, MMU_DATA_LOAD, 1, + MMU_DATA_LOAD, 1, GETPC()); + + /* Load if page supports tags. */ + if (mem) { + rtag = load_tag1(ptr, mem); + } + + return address_with_allocation_tag(xt, rtag); +} + +static void check_tag_aligned(CPUARMState *env, uint64_t ptr, uintptr_t ra) +{ + if (unlikely(!QEMU_IS_ALIGNED(ptr, TAG_GRANULE))) { + arm_cpu_do_unaligned_access(env_cpu(env), ptr, MMU_DATA_STORE, + cpu_mmu_index(env, false), ra); + g_assert_not_reached(); + } +} + +/* For use in a non-parallel context, store to the given nibble. */ +static void store_tag1(uint64_t ptr, uint8_t *mem, int tag) +{ + int ofs = extract32(ptr, LOG2_TAG_GRANULE, 1) * 4; + *mem = deposit32(*mem, ofs, 4, tag); +} + +/* For use in a parallel context, atomically store to the given nibble. */ +static void store_tag1_parallel(uint64_t ptr, uint8_t *mem, int tag) +{ + int ofs = extract32(ptr, LOG2_TAG_GRANULE, 1) * 4; + uint8_t old = atomic_read(mem); + + while (1) { + uint8_t new = deposit32(old, ofs, 4, tag); + uint8_t cmp = atomic_cmpxchg(mem, old, new); + if (likely(cmp == old)) { + return; + } + old = cmp; + } +} + +typedef void stg_store1(uint64_t, uint8_t *, int); + +static inline void do_stg(CPUARMState *env, uint64_t ptr, uint64_t xt, + uintptr_t ra, stg_store1 store1) +{ + int mmu_idx = cpu_mmu_index(env, false); + uint8_t *mem; + + check_tag_aligned(env, ptr, ra); + + /* Trap if accessing an invalid page. */ + mem = allocation_tag_mem(env, mmu_idx, ptr, MMU_DATA_STORE, TAG_GRANULE, + MMU_DATA_STORE, 1, ra); + + /* Store if page supports tags. */ + if (mem) { + store1(ptr, mem, allocation_tag_from_addr(xt)); + } +} + +void HELPER(stg)(CPUARMState *env, uint64_t ptr, uint64_t xt) +{ + do_stg(env, ptr, xt, GETPC(), store_tag1); +} + +void HELPER(stg_parallel)(CPUARMState *env, uint64_t ptr, uint64_t xt) +{ + do_stg(env, ptr, xt, GETPC(), store_tag1_parallel); +} + +void HELPER(stg_stub)(CPUARMState *env, uint64_t ptr) +{ + int mmu_idx = cpu_mmu_index(env, false); + uintptr_t ra = GETPC(); + + check_tag_aligned(env, ptr, ra); + probe_write(env, ptr, TAG_GRANULE, mmu_idx, ra); +} + +static inline void do_st2g(CPUARMState *env, uint64_t ptr, uint64_t xt, + uintptr_t ra, stg_store1 store1) +{ + int mmu_idx = cpu_mmu_index(env, false); + int tag = allocation_tag_from_addr(xt); + uint8_t *mem1, *mem2; + + check_tag_aligned(env, ptr, ra); + + /* + * Trap if accessing an invalid page(s). + * This takes priority over !allocation_tag_access_enabled. + */ + if (ptr & TAG_GRANULE) { + /* Two stores unaligned mod TAG_GRANULE*2 -- modify two bytes. */ + mem1 = allocation_tag_mem(env, mmu_idx, ptr, MMU_DATA_STORE, + TAG_GRANULE, MMU_DATA_STORE, 1, ra); + mem2 = allocation_tag_mem(env, mmu_idx, ptr + TAG_GRANULE, + MMU_DATA_STORE, TAG_GRANULE, + MMU_DATA_STORE, 1, ra); + + /* Store if page(s) support tags. */ + if (mem1) { + store1(TAG_GRANULE, mem1, tag); + } + if (mem2) { + store1(0, mem2, tag); + } + } else { + /* Two stores aligned mod TAG_GRANULE*2 -- modify one byte. */ + mem1 = allocation_tag_mem(env, mmu_idx, ptr, MMU_DATA_STORE, + 2 * TAG_GRANULE, MMU_DATA_STORE, 1, ra); + if (mem1) { + tag |= tag << 4; + atomic_set(mem1, tag); + } + } +} + +void HELPER(st2g)(CPUARMState *env, uint64_t ptr, uint64_t xt) +{ + do_st2g(env, ptr, xt, GETPC(), store_tag1); +} + +void HELPER(st2g_parallel)(CPUARMState *env, uint64_t ptr, uint64_t xt) +{ + do_st2g(env, ptr, xt, GETPC(), store_tag1_parallel); +} + +void HELPER(st2g_stub)(CPUARMState *env, uint64_t ptr) +{ + int mmu_idx = cpu_mmu_index(env, false); + uintptr_t ra = GETPC(); + int in_page = -(ptr | TARGET_PAGE_MASK); + + check_tag_aligned(env, ptr, ra); + + if (likely(in_page >= 2 * TAG_GRANULE)) { + probe_write(env, ptr, 2 * TAG_GRANULE, mmu_idx, ra); + } else { + probe_write(env, ptr, TAG_GRANULE, mmu_idx, ra); + probe_write(env, ptr + TAG_GRANULE, TAG_GRANULE, mmu_idx, ra); + } +} diff --git a/target/arm/translate-a64.c b/target/arm/translate-a64.c index b032829194..0725c2ca07 100644 --- a/target/arm/translate-a64.c +++ b/target/arm/translate-a64.c @@ -3689,6 +3689,153 @@ static void disas_ldst_single_struct(DisasContext *s, uint32_t insn) } } +/* + * Load/Store memory tags + * + * 31 30 29 24 22 21 12 10 5 0 + * +-----+-------------+-----+---+------+-----+------+------+ + * | 1 1 | 0 1 1 0 0 1 | op1 | 1 | imm9 | op2 | Rn | Rt | + * +-----+-------------+-----+---+------+-----+------+------+ + */ +static void disas_ldst_tag(DisasContext *s, uint32_t insn) +{ + int rt = extract32(insn, 0, 5); + int rn = extract32(insn, 5, 5); + uint64_t offset = sextract64(insn, 12, 9) << LOG2_TAG_GRANULE; + int op2 = extract32(insn, 10, 2); + int op1 = extract32(insn, 22, 2); + bool is_load = false, is_pair = false, is_zero = false; + int index = 0; + TCGv_i64 addr, clean_addr, tcg_rt; + + /* We checked insn bits [29:24,21] in the caller. */ + if (extract32(insn, 30, 2) != 3) { + goto do_unallocated; + } + + /* + * @index is a tri-state variable which has 3 states: + * < 0 : post-index, writeback + * = 0 : signed offset + * > 0 : pre-index, writeback + */ + switch (op1) { + case 0: /* STG */ + if (op2 != 0) { + /* STG */ + index = op2 - 2; + break; + } + goto do_unallocated; + case 1: + if (op2 != 0) { + /* STZG */ + is_zero = true; + index = op2 - 2; + } else { + /* LDG */ + is_load = true; + } + break; + case 2: + if (op2 != 0) { + /* ST2G */ + is_pair = true; + index = op2 - 2; + break; + } + goto do_unallocated; + case 3: + if (op2 != 0) { + /* STZ2G */ + is_pair = is_zero = true; + index = op2 - 2; + break; + } + goto do_unallocated; + + default: + do_unallocated: + unallocated_encoding(s); + return; + } + + if (!dc_isar_feature(aa64_mte_insn_reg, s)) { + goto do_unallocated; + } + + if (rn == 31) { + gen_check_sp_alignment(s); + } + + addr = read_cpu_reg_sp(s, rn, true); + if (index >= 0) { + /* pre-index or signed offset */ + tcg_gen_addi_i64(addr, addr, offset); + } + + if (is_load) { + tcg_rt = cpu_reg(s, rt); + if (s->ata) { + gen_helper_ldg(tcg_rt, cpu_env, addr, tcg_rt); + } else { + clean_addr = clean_data_tbi(s, addr); + gen_probe_access(s, clean_addr, MMU_DATA_LOAD, MO_8); + gen_address_with_allocation_tag0(tcg_rt, addr); + } + } else { + tcg_rt = cpu_reg_sp(s, rt); + if (!s->ata) { + /* + * For STG and ST2G, we need to check alignment and probe memory. + * TODO: For STZG and STZ2G, we could rely on the stores below, + * at least for system mode; user-only won't enforce alignment. + */ + if (is_pair) { + gen_helper_st2g_stub(cpu_env, addr); + } else { + gen_helper_stg_stub(cpu_env, addr); + } + } else if (tb_cflags(s->base.tb) & CF_PARALLEL) { + if (is_pair) { + gen_helper_st2g_parallel(cpu_env, addr, tcg_rt); + } else { + gen_helper_stg_parallel(cpu_env, addr, tcg_rt); + } + } else { + if (is_pair) { + gen_helper_st2g(cpu_env, addr, tcg_rt); + } else { + gen_helper_stg(cpu_env, addr, tcg_rt); + } + } + } + + if (is_zero) { + TCGv_i64 clean_addr = clean_data_tbi(s, addr); + TCGv_i64 tcg_zero = tcg_const_i64(0); + int mem_index = get_mem_index(s); + int i, n = (1 + is_pair) << LOG2_TAG_GRANULE; + + tcg_gen_qemu_st_i64(tcg_zero, clean_addr, mem_index, + MO_Q | MO_ALIGN_16); + for (i = 8; i < n; i += 8) { + tcg_gen_addi_i64(clean_addr, clean_addr, 8); + tcg_gen_qemu_st_i64(tcg_zero, clean_addr, mem_index, MO_Q); + } + tcg_temp_free_i64(tcg_zero); + } + + if (index != 0) { + /* pre-index or post-index */ + if (index < 0) { + /* post-index */ + tcg_gen_addi_i64(addr, addr, offset); + } + tcg_gen_mov_i64(cpu_reg_sp(s, rn), addr); + } +} + /* Loads and stores */ static void disas_ldst(DisasContext *s, uint32_t insn) { @@ -3713,13 +3860,14 @@ static void disas_ldst(DisasContext *s, uint32_t insn) case 0x0d: /* AdvSIMD load/store single structure */ disas_ldst_single_struct(s, insn); break; - case 0x19: /* LDAPR/STLR (unscaled immediate) */ - if (extract32(insn, 10, 2) != 0 || - extract32(insn, 21, 1) != 0) { + case 0x19: + if (extract32(insn, 21, 1) != 0) { + disas_ldst_tag(s, insn); + } else if (extract32(insn, 10, 2) == 0) { + disas_ldst_ldapr_stlr(s, insn); + } else { unallocated_encoding(s); - break; } - disas_ldst_ldapr_stlr(s, insn); break; default: unallocated_encoding(s); From patchwork Wed Jun 3 01:12:51 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 187422 Delivered-To: patch@linaro.org Received: by 2002:a92:cf06:0:0:0:0:0 with SMTP id c6csp75591ilo; Tue, 2 Jun 2020 18:23:52 -0700 (PDT) X-Google-Smtp-Source: ABdhPJwusxwGW8FingLApDfj1GqD3zq7/0+yxydxQ9uwbXQ6rwb17Na3YTFvd+B/vZ1iN4FJ6yzy X-Received: by 2002:a25:be05:: with SMTP id h5mr13246341ybk.431.1591147432672; Tue, 02 Jun 2020 18:23:52 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1591147432; cv=none; d=google.com; s=arc-20160816; b=jdIR83y3QWvWZ9uKlmD4pXirlh+Bh/w48f4nDD3Fg/IX8KWixrWGcNsHnf4XY46nr2 XgS6M2laUsoSuo0DrnxnEwhsMP/6RBM09OrM3eKJ2h4Zs8pj81zbbR2SNC0Q/Mhb7mwm Jbl4zb/dnzhSA6rw/XoUEss1my1vqS/uMXElrdfA3POFYQ7zgnGWemD1Bf4koNx+P5pv SRc73bghGu8arAN3JWcjDdXX+qkknSgSb68PnGUj07rP2XyxK3wglI+v7RXpC2PUy9d5 bXWsPbTzz9eOpkxu0k3x6WpysGkB0KAzkDwyFkqZcytHCiY/7QK8Vf7AUMMSrxERx7aq 7lDQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=lQUCdDSymie/7dQ1iavooQ03Mfu1YJhrKa9eVrF8VLk=; b=VS+6F5CkCs7vgUxV0AjazCbsNTAbqaeLJPCGJdZASGIyTES7aART+rBLXiEL9jflUy 3pCs1sRVjExUi2CWn7EbK3/3iyxhh0QW57aWUpWNy5yeVqoqDCgJWdRYMU17ldqrNhpR AApvCLI/l22vkgQPD9FxfOO1XU9lUlqMaMaGbi9/53r/EOTsdWHoNWre/F3dEbQyZjCE 9MDp5BofIJPEOcbtBF4i89gVkaYJj9WHp208Un6B5SWQ2tY1ZnumqGKymXfcJ4W6bbOK BP3J2dP0g8ir5jC/LVgEpJ++/k74WVK4mG51gFGWXQMiT9sfjItPDg5ztISRThSzFMsA 29Tg== ARC-Authentication-Results: i=1; mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b="U/q0uTVo"; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id r9si351737ybb.182.2020.06.02.18.23.52 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 02 Jun 2020 18:23:52 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b="U/q0uTVo"; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:36230 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jgI8G-00069D-3p for patch@linaro.org; Tue, 02 Jun 2020 21:23:52 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:48734) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jgHyV-00040p-H2 for qemu-devel@nongnu.org; Tue, 02 Jun 2020 21:13:47 -0400 Received: from mail-pf1-x443.google.com ([2607:f8b0:4864:20::443]:46895) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1jgHyP-0003Zm-CQ for qemu-devel@nongnu.org; Tue, 02 Jun 2020 21:13:47 -0400 Received: by mail-pf1-x443.google.com with SMTP id b16so395832pfi.13 for ; Tue, 02 Jun 2020 18:13:40 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=lQUCdDSymie/7dQ1iavooQ03Mfu1YJhrKa9eVrF8VLk=; b=U/q0uTVoFwQhjnL9XwKPSwInTYw7BqC/t+ZGwi4JRUW/sPbFLP5A5TDNoSz6MTkBvb sgs0BUjzNzoB8LmB8YKm/6ZQJs/NGKZ+1R5pGrovVN+iMAKzyjFLFGDQaDlqEYiFZGj/ ib41y5livE10QOUJLkkGuFlUPOscTndPf6Povg7tGl03wj5imEdYaYEDAjwoa8/Qgs+c ywnz91TtgvzVIDXjPaILVMYrIIfqIM66v3ycbganuzBIm/m2X3X7awbcqMbLf+Ltn8Kp +6sFQz0G9xLrXbRou2x9BeEVGzd7+JK5o5und6fnd6pYbBljpg89+5FDDYmZDebydoQw iRpw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=lQUCdDSymie/7dQ1iavooQ03Mfu1YJhrKa9eVrF8VLk=; b=KvC7dZA5w1m+1HEk/jgqAuX6HwRz2GSFTJspohJmeDsWoAjb40Xxs9u4MY8HdcWVWC 6CqenacNKxDnYw1lcfwT1S5sPq4InzHFZGaNcLK9ZyKEBdQ6OSWSJNO/D0KNEbzO1Xqp EGYzvFdvMPHiQJtOeVeuUDdEK0Bvkbd6HFF5RfI6R/HpgSs2PR9kC8btS7IpMgtDG2p4 Ypz+JplCBmOXVM6rCl6NiazlrteGvnQa9pHmc/verMyF6GpUklsdNS77g1J1G9skd8qz vahMLdJd61Brl7NEA3Dl1+HRv4wq1cYX+iclhs+arQKxkwr91FawVdH7XaevExd3p5xl TRQg== X-Gm-Message-State: AOAM5315FxOSAl2DCmR30tKF9wt53iRvsL4ZuARW523Opl/Sm0kpYEZ1 X0kPOPBmKzwYgmI2A8tqmSQ6XP55aiY= X-Received: by 2002:a63:348:: with SMTP id 69mr25417768pgd.68.1591146819477; Tue, 02 Jun 2020 18:13:39 -0700 (PDT) Received: from localhost.localdomain (174-21-143-238.tukw.qwest.net. [174.21.143.238]) by smtp.gmail.com with ESMTPSA id 3sm290067pfe.85.2020.06.02.18.13.38 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 02 Jun 2020 18:13:38 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v7 16/42] target/arm: Implement the STGP instruction Date: Tue, 2 Jun 2020 18:12:51 -0700 Message-Id: <20200603011317.473934-17-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200603011317.473934-1-richard.henderson@linaro.org> References: <20200603011317.473934-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::443; envelope-from=richard.henderson@linaro.org; helo=mail-pf1-x443.google.com X-detected-operating-system: by eggs.gnu.org: No matching host in p0f cache. That's all we know. X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: peter.maydell@linaro.org, qemu-arm@nongnu.org, steplong@quicinc.com Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" Reviewed-by: Peter Maydell Signed-off-by: Richard Henderson --- v3: Handle atomicity, require pre-cleaned address. v6: Fix constant offset shift, non-checked address, use pre-computed ata. --- target/arm/translate-a64.c | 29 ++++++++++++++++++++++++++--- 1 file changed, 26 insertions(+), 3 deletions(-) -- 2.25.1 diff --git a/target/arm/translate-a64.c b/target/arm/translate-a64.c index 0725c2ca07..2fc070ab90 100644 --- a/target/arm/translate-a64.c +++ b/target/arm/translate-a64.c @@ -2681,7 +2681,7 @@ static void disas_ld_lit(DisasContext *s, uint32_t insn) * +-----+-------+---+---+-------+---+-------+-------+------+------+ * * opc: LDP/STP/LDNP/STNP 00 -> 32 bit, 10 -> 64 bit - * LDPSW 01 + * LDPSW/STGP 01 * LDP/STP/LDNP/STNP (SIMD) 00 -> 32 bit, 01 -> 64 bit, 10 -> 128 bit * V: 0 -> GPR, 1 -> Vector * idx: 00 -> signed offset with non-temporal hint, 01 -> post-index, @@ -2706,6 +2706,7 @@ static void disas_ldst_pair(DisasContext *s, uint32_t insn) bool is_signed = false; bool postindex = false; bool wback = false; + bool set_tag = false; TCGv_i64 clean_addr, dirty_addr; @@ -2718,6 +2719,14 @@ static void disas_ldst_pair(DisasContext *s, uint32_t insn) if (is_vector) { size = 2 + opc; + } else if (opc == 1 && !is_load) { + /* STGP */ + if (!dc_isar_feature(aa64_mte_insn_reg, s) || index == 0) { + unallocated_encoding(s); + return; + } + size = 3; + set_tag = true; } else { size = 2 + extract32(opc, 1, 1); is_signed = extract32(opc, 0, 1); @@ -2758,7 +2767,7 @@ static void disas_ldst_pair(DisasContext *s, uint32_t insn) return; } - offset <<= size; + offset <<= (set_tag ? LOG2_TAG_GRANULE : size); if (rn == 31) { gen_check_sp_alignment(s); @@ -2768,8 +2777,22 @@ static void disas_ldst_pair(DisasContext *s, uint32_t insn) if (!postindex) { tcg_gen_addi_i64(dirty_addr, dirty_addr, offset); } - clean_addr = clean_data_tbi(s, dirty_addr); + if (set_tag) { + if (!s->ata) { + /* + * TODO: We could rely on the stores below, at least for + * system mode, if we arrange to add MO_ALIGN_16. + */ + gen_helper_stg_stub(cpu_env, dirty_addr); + } else if (tb_cflags(s->base.tb) & CF_PARALLEL) { + gen_helper_stg_parallel(cpu_env, dirty_addr, dirty_addr); + } else { + gen_helper_stg(cpu_env, dirty_addr, dirty_addr); + } + } + + clean_addr = clean_data_tbi(s, dirty_addr); if (is_vector) { if (is_load) { do_fp_ld(s, rt, clean_addr, size); From patchwork Wed Jun 3 01:12:52 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 187413 Delivered-To: patch@linaro.org Received: by 2002:a92:cf06:0:0:0:0:0 with SMTP id c6csp73345ilo; Tue, 2 Jun 2020 18:19:59 -0700 (PDT) X-Google-Smtp-Source: ABdhPJzfXFVQJCPZJDX8DCd/8CAOhmHx/ZhZFUiuvxScDPrbh3ShqveKxdlAMG5sstR+g0CgvKkJ X-Received: by 2002:a25:d8d5:: with SMTP id p204mr33188417ybg.28.1591147199020; Tue, 02 Jun 2020 18:19:59 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1591147199; cv=none; d=google.com; s=arc-20160816; b=qx0va6KbgX5hTgib0oknp/V0H+o/qWsxnYDuo0R5bgMsDh2/iL7lNY7JrDMT9sxwsq IzvO5wUpWgQ1Q01TGz4WoMdg8F2L+F05cUyLDVZOPW4AeTWhcB3EXMwzWjauNLTp1fxv kzrTW91yl9oHTdS0IanvYI3zg0uzqJwXOB0RSuGmGkN3ceAEOG3VQvFXJNySeVL2SuJN giXQsOseblsrkHZeNgKQm3/1qoMnRHRfdfbp6H/WRuODWpB/YjVkI+KaadoVlOK2SitN DLqSQS1TPteESIRip3QcakFep4GuGqCUUl7WIV6EciVqX/Al7wGgS/BWaGyHrcaGBQI9 fSFw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=C4qDuL752EiiSk3nsMH6iLKPjyLVOQMWVDOC2KEux/s=; b=aUM2jpYYhrb7/2rkNFMqmQOpK9+kJHA/RIgQbYPMQhoR0k18treMxYAYmncO4rkPjN uhs5OpKoa8dL3VkF0iRk1THLZTc0ngwpVzMZ1i2rFBhdLRRC4BA5vXWQjNy7XzE5UCkQ IRNGUhFh6236FGvPjemZ0ESADzTPzy8ddHa1tyjyztHHMl6AwVCAaFlIBUR4oXrZq5Ds eq0Mijs0YeCid41eiDQuOn8gQdITB6f8bq70m4YKtIHWtmY8hS+LxCDmQGUxKWahCIv7 Fs20uMaLIVK9A8RvLjfLC4L1K0qftCVVykx/Q8WNUnmamwkpNQ1X77vAHS6gATAsIp/3 WPjw== ARC-Authentication-Results: i=1; mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=AkB3uzpT; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id n63si349076ybc.276.2020.06.02.18.19.58 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 02 Jun 2020 18:19:59 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=AkB3uzpT; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:47490 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jgI4U-0007bs-4e for patch@linaro.org; Tue, 02 Jun 2020 21:19:58 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:48744) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jgHyW-00042o-7C for qemu-devel@nongnu.org; Tue, 02 Jun 2020 21:13:48 -0400 Received: from mail-pl1-x641.google.com ([2607:f8b0:4864:20::641]:33271) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1jgHyR-0003a2-5M for qemu-devel@nongnu.org; Tue, 02 Jun 2020 21:13:47 -0400 Received: by mail-pl1-x641.google.com with SMTP id t7so209481plr.0 for ; Tue, 02 Jun 2020 18:13:41 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=C4qDuL752EiiSk3nsMH6iLKPjyLVOQMWVDOC2KEux/s=; b=AkB3uzpTGuxEieVSjMGtHxWwJlZmAJJoRb0uBkEqwza+LNGxNHjYgyjCmUNVa0gd6t iSKf+2SNR57eagpHllanngZWVqzWfV0cRdc91jOJZt529NQFne0GYs2+0oyqWA9K5wz9 P0KZ6zE0EXyOJu5kCzNc/aJ/DyJ6avuxVcLBQZOpX3iYW5qqKGc4qC/nO3wIbpsmzx1W IGxzJK0uYMPZl3Y8AmsHqoxsmxLOeD0FAt5lGq5QYQQyCZVlaCMxt5aTMOmmyUDbAUkn DkanfPaCxZRqTS2Mc05PSHwKll2yGR3r80uJpQqCIKSPk8IzrML90PzwP+seXFevG7fa Na8g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=C4qDuL752EiiSk3nsMH6iLKPjyLVOQMWVDOC2KEux/s=; b=SREb6Ry3pcDxmIdd3ndAYx5baiVUZN8Px/gTEUnw/tyfsDi+DwDahaYYgC4BnhRzcH BVQlHMSb6pPdiU1xebubxFmfMw45kvhh6tZgYhSqfpi9RZgWJkMQpO/pRFG26Mf8y8F6 7uD5YlpXjTIhM3+mZ9J22QmFHZc2LNGeL4XlUiK2i1ENhAcFSVtf1GpEfKfCa3A8mWH2 iLCYgm207Vvzmp+64aWR9jm0wbaurj5NpRPLNyyksgl8+gvJZKBjDHEkhFIM0slHh+LT 3GiYpXc/eoclmF7QruHoGcruRMKFDBmvLwnaCb1HMg1TCWxkRfPr/cVRGSBTAnX3HyPH kung== X-Gm-Message-State: AOAM530lj3HWzjpVIxDZhwBWE6N6aA+KtH3QohPRrHAhOrmOuG7PbsRk kH3Whdt/vYMAUtty0TDE4SxPpRpleC8= X-Received: by 2002:a17:902:9a43:: with SMTP id x3mr27532907plv.332.1591146820716; Tue, 02 Jun 2020 18:13:40 -0700 (PDT) Received: from localhost.localdomain (174-21-143-238.tukw.qwest.net. [174.21.143.238]) by smtp.gmail.com with ESMTPSA id 3sm290067pfe.85.2020.06.02.18.13.39 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 02 Jun 2020 18:13:40 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v7 17/42] target/arm: Restrict the values of DCZID.BS under TCG Date: Tue, 2 Jun 2020 18:12:52 -0700 Message-Id: <20200603011317.473934-18-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200603011317.473934-1-richard.henderson@linaro.org> References: <20200603011317.473934-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::641; envelope-from=richard.henderson@linaro.org; helo=mail-pl1-x641.google.com X-detected-operating-system: by eggs.gnu.org: No matching host in p0f cache. That's all we know. X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: peter.maydell@linaro.org, qemu-arm@nongnu.org, steplong@quicinc.com Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" We can simplify our DC_ZVA if we recognize that the largest BS that we actually use in system mode is 64. Let us just assert that it fits within TARGET_PAGE_SIZE. For DC_GVA and STZGM, we want to be able to write whole bytes of tag memory, so assert that BS is >= 2 * TAG_GRANULE, or 32. Signed-off-by: Richard Henderson --- target/arm/cpu.c | 24 ++++++++++++++++++++++++ 1 file changed, 24 insertions(+) -- 2.25.1 Reviewed-by: Peter Maydell diff --git a/target/arm/cpu.c b/target/arm/cpu.c index 0f1a46f531..8e6d4371cb 100644 --- a/target/arm/cpu.c +++ b/target/arm/cpu.c @@ -1754,6 +1754,30 @@ static void arm_cpu_realizefn(DeviceState *dev, Error **errp) } #endif + if (tcg_enabled()) { + int dcz_blocklen = 4 << cpu->dcz_blocksize; + + /* + * We only support DCZ blocklen that fits on one page. + * + * Architectually this is always true. However TARGET_PAGE_SIZE + * is variable and, for compatibility with -machine virt-2.7, + * is only 1KiB, as an artifact of legacy ARMv5 subpage support. + * But even then, while the largest architectural DCZ blocklen + * is 2KiB, no cpu actually uses such a large blocklen. + */ + assert(dcz_blocklen <= TARGET_PAGE_SIZE); + + /* + * We only support DCZ blocksize >= 2*TAG_GRANULE, which is to say + * both nibbles of each byte storing tag data may be written at once. + * Since TAG_GRANULE is 16, this means that blocklen must be >= 32. + */ + if (cpu_isar_feature(aa64_mte, cpu)) { + assert(dcz_blocklen >= 2 * TAG_GRANULE); + } + } + qemu_init_vcpu(cs); cpu_reset(cs); From patchwork Wed Jun 3 01:12:53 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 187410 Delivered-To: patch@linaro.org Received: by 2002:a92:cf06:0:0:0:0:0 with SMTP id c6csp72471ilo; Tue, 2 Jun 2020 18:18:30 -0700 (PDT) X-Google-Smtp-Source: ABdhPJxpSRrx3D2qOyubJAcZm6FnhqqLz8To/ibd4MvGk5jAZuDfWthx72udI3qejJ0TOFB5/j6j X-Received: by 2002:a25:46c6:: with SMTP id t189mr44699181yba.486.1591147110812; Tue, 02 Jun 2020 18:18:30 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1591147110; cv=none; d=google.com; s=arc-20160816; b=zO9dOEMgSewFr/8xqpGLtGXYeYzR5jyFFKDSWS+CE8dzs7f3LGDfN+7bkOPnpi5vfH 4QvJWXIF1UTv44g8nGEFKL21KdTFccKunFt/c1roPXlpETAhFqSeU9FqKwhCMOq/AIfO CBpK/eMFJIFgliLvWVzNmGhnzhNY0OQNHgfAdKSu9ZRFv6BblgRpalnkkOtrwIfttCVB h/GvhbUj6Y1Ya8IU3q0iNPv2aS3VGbMPpEMF4VvlL4dOq3uh4660GNtOdq6pJ/8+Kr3w GXYHliI5OUnucKrcAXr9hDbtDjipiYOF10vJRtoEU4KWcw2LZRn6yZCFijfg+pUxYymg dAUA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=o1+JkLvr5wpvNs6tBAnE0GN83gyiuk70hCD7c+aSnks=; b=H2wzXJ8or1TkX8tpI9jpWawdjUC9ppWQd5mxYv6uPK1v5MSFT3x0+fbMbiBkm3hyuG TCv3MyRyELF1Q++85r74VbJA+E7yOo2Gqlp7KUnF5hh57EZGPWnZgy/yDOrep6dvRHPF Pk0lXHHLbGvLDa12ZkyUuOENRO2BAh0mE+Fw1Z0R1K0dcu58InP80muGuK3eA53Vslk0 xMOZLKSFVrP0z5O1UQQ6QC54NFmBEKd2h1O1WAFVg/I9pwxruvl0u6hxwaJbZRBh6s8w ofbPgyfxV8sWV3phuR3DXR8gxDLiqKplBxwFaeZaWwN3TP4uG8ljOJfCOoZYlV3zVDmt 3kPQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=VycdNoSd; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id l143si304488ybf.155.2020.06.02.18.18.30 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 02 Jun 2020 18:18:30 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=VycdNoSd; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:39150 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jgI34-0004LH-4h for patch@linaro.org; Tue, 02 Jun 2020 21:18:30 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:48752) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jgHyW-00044w-V2 for qemu-devel@nongnu.org; Tue, 02 Jun 2020 21:13:48 -0400 Received: from mail-pg1-x541.google.com ([2607:f8b0:4864:20::541]:42683) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1jgHyT-0003aF-0J for qemu-devel@nongnu.org; Tue, 02 Jun 2020 21:13:48 -0400 Received: by mail-pg1-x541.google.com with SMTP id e9so532640pgo.9 for ; Tue, 02 Jun 2020 18:13:43 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=o1+JkLvr5wpvNs6tBAnE0GN83gyiuk70hCD7c+aSnks=; b=VycdNoSdZyQzr3hcpjbmuYAreGJnWyoVh2a/xWLcsMnXjePsroXZii2ddG3Q0rjAVb PuVK6s8IiwGtLvNEsiXU1LC5v73AjXNRRr3FBDFJaWDUCLzsjSKl121cjHCopc/4y4OV ezGhVp/vZxMdm9QvHVggbPvTR/rdDBY1amJcShxqJB1hWYT07wMSNLS8zZfszTPAEMav jejRvlD7caBUEzwaKrJ45OFQMMCr0nY5xg1Ld9nf6LB/UO+u2n8QRQ/6IpOSfAkR8I5M tcOkJHA8XSxCkR7ERjjv2kU3wf0KfQac7i5VQKYk8BwrxR7HqwgmQ7lNfwhEUSj+vuw8 enCg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=o1+JkLvr5wpvNs6tBAnE0GN83gyiuk70hCD7c+aSnks=; b=g21BePuBDSpYUez1spc+tlIDlDxY8Vv6CrX+NXCgwL5s30PQ2o4C28zeWracGW37fb Cr/epPUw5aFjAljCuyjHkHYkDBxmQOxoZvVwD5ZvuCCBfUgI3T6Ej7NfW7NsAVoSArrv RaL9utK/rfZOjDmqsolK14xQxMMZUXFmnlxL3An2hsnM17oviPwIdjMiJB+uLRmQG6eI WRtw8TMC3ET8kmtpXvIaekS80vnzvHl3OvErTcegUcHGJv659i+04AddtiK5/J27jRmd Im9vx4Hz1VnhK3FwgvRV7C+vsqJcpPjBD+P81Iy8GwQSoaSlDO0z+AM8Psf/CvhCUo3z ETqg== X-Gm-Message-State: AOAM531uBukh6Kd8qlk/LhU/h6jbCjtrkxmlhJMAHGbQ1s2/WSgQd3qj 21YkjDKavgeg2mBw7bg8xzZtfRShONA= X-Received: by 2002:a63:f54f:: with SMTP id e15mr14570507pgk.234.1591146822101; Tue, 02 Jun 2020 18:13:42 -0700 (PDT) Received: from localhost.localdomain (174-21-143-238.tukw.qwest.net. [174.21.143.238]) by smtp.gmail.com with ESMTPSA id 3sm290067pfe.85.2020.06.02.18.13.40 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 02 Jun 2020 18:13:41 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v7 18/42] target/arm: Simplify DC_ZVA Date: Tue, 2 Jun 2020 18:12:53 -0700 Message-Id: <20200603011317.473934-19-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200603011317.473934-1-richard.henderson@linaro.org> References: <20200603011317.473934-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::541; envelope-from=richard.henderson@linaro.org; helo=mail-pg1-x541.google.com X-detected-operating-system: by eggs.gnu.org: No matching host in p0f cache. That's all we know. X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: peter.maydell@linaro.org, qemu-arm@nongnu.org, steplong@quicinc.com Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" Now that we know that the operation is on a single page, we need not loop over pages while probing. Signed-off-by: Richard Henderson --- target/arm/helper-a64.c | 94 +++++++++++------------------------------ 1 file changed, 25 insertions(+), 69 deletions(-) -- 2.25.1 Reviewed-by: Peter Maydell diff --git a/target/arm/helper-a64.c b/target/arm/helper-a64.c index bc0649a44a..60a04dc880 100644 --- a/target/arm/helper-a64.c +++ b/target/arm/helper-a64.c @@ -1119,85 +1119,41 @@ void HELPER(dc_zva)(CPUARMState *env, uint64_t vaddr_in) * (which matches the usual QEMU behaviour of not implementing either * alignment faults or any memory attribute handling). */ + int blocklen = 4 << env_archcpu(env)->dcz_blocksize; + uint64_t vaddr = vaddr_in & -blocklen; + int mmu_idx = cpu_mmu_index(env, false); + void *mem; - ARMCPU *cpu = env_archcpu(env); - uint64_t blocklen = 4 << cpu->dcz_blocksize; - uint64_t vaddr = vaddr_in & ~(blocklen - 1); + /* + * Trapless lookup. In addition to actual invalid page, may + * return NULL for I/O, watchpoints, clean pages, etc. + */ + mem = tlb_vaddr_to_host(env, vaddr, MMU_DATA_STORE, mmu_idx); #ifndef CONFIG_USER_ONLY - { + if (unlikely(!mem)) { + uintptr_t ra = GETPC(); + /* - * Slightly awkwardly, QEMU's TARGET_PAGE_SIZE may be less than - * the block size so we might have to do more than one TLB lookup. - * We know that in fact for any v8 CPU the page size is at least 4K - * and the block size must be 2K or less, but TARGET_PAGE_SIZE is only - * 1K as an artefact of legacy v5 subpage support being present in the - * same QEMU executable. So in practice the hostaddr[] array has - * two entries, given the current setting of TARGET_PAGE_BITS_MIN. + * Trap if accessing an invalid page. DC_ZVA requires that we supply + * the original pointer for an invalid page. But watchpoints require + * that we probe the actual space. So do both. */ - int maxidx = DIV_ROUND_UP(blocklen, TARGET_PAGE_SIZE); - void *hostaddr[DIV_ROUND_UP(2 * KiB, 1 << TARGET_PAGE_BITS_MIN)]; - int try, i; - unsigned mmu_idx = cpu_mmu_index(env, false); - TCGMemOpIdx oi = make_memop_idx(MO_UB, mmu_idx); + (void) probe_write(env, vaddr_in, 1, mmu_idx, ra); + mem = probe_write(env, vaddr, blocklen, mmu_idx, ra); - assert(maxidx <= ARRAY_SIZE(hostaddr)); - - for (try = 0; try < 2; try++) { - - for (i = 0; i < maxidx; i++) { - hostaddr[i] = tlb_vaddr_to_host(env, - vaddr + TARGET_PAGE_SIZE * i, - 1, mmu_idx); - if (!hostaddr[i]) { - break; - } - } - if (i == maxidx) { - /* - * If it's all in the TLB it's fair game for just writing to; - * we know we don't need to update dirty status, etc. - */ - for (i = 0; i < maxidx - 1; i++) { - memset(hostaddr[i], 0, TARGET_PAGE_SIZE); - } - memset(hostaddr[i], 0, blocklen - (i * TARGET_PAGE_SIZE)); - return; - } + if (unlikely(!mem)) { /* - * OK, try a store and see if we can populate the tlb. This - * might cause an exception if the memory isn't writable, - * in which case we will longjmp out of here. We must for - * this purpose use the actual register value passed to us - * so that we get the fault address right. + * The only remaining reason for mem == NULL is I/O. + * Just do a series of byte writes as the architecture demands. */ - helper_ret_stb_mmu(env, vaddr_in, 0, oi, GETPC()); - /* Now we can populate the other TLB entries, if any */ - for (i = 0; i < maxidx; i++) { - uint64_t va = vaddr + TARGET_PAGE_SIZE * i; - if (va != (vaddr_in & TARGET_PAGE_MASK)) { - helper_ret_stb_mmu(env, va, 0, oi, GETPC()); - } + for (int i = 0; i < blocklen; i++) { + cpu_stb_mmuidx_ra(env, vaddr + i, 0, mmu_idx, ra); } - } - - /* - * Slow path (probably attempt to do this to an I/O device or - * similar, or clearing of a block of code we have translations - * cached for). Just do a series of byte writes as the architecture - * demands. It's not worth trying to use a cpu_physical_memory_map(), - * memset(), unmap() sequence here because: - * + we'd need to account for the blocksize being larger than a page - * + the direct-RAM access case is almost always going to be dealt - * with in the fastpath code above, so there's no speed benefit - * + we would have to deal with the map returning NULL because the - * bounce buffer was in use - */ - for (i = 0; i < blocklen; i++) { - helper_ret_stb_mmu(env, vaddr + i, 0, oi, GETPC()); + return; } } -#else - memset(g2h(vaddr), 0, blocklen); #endif + + memset(mem, 0, blocklen); } From patchwork Wed Jun 3 01:12:54 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 187418 Delivered-To: patch@linaro.org Received: by 2002:a92:cf06:0:0:0:0:0 with SMTP id c6csp74549ilo; Tue, 2 Jun 2020 18:21:58 -0700 (PDT) X-Google-Smtp-Source: ABdhPJxEm6nETHDpti2ld8K6AG4qk14jlgd9wT1ldnqjA7m7TFpFueRo7DyXZnczvsy9cgHJ0JYw X-Received: by 2002:a25:c683:: with SMTP id k125mr22256799ybf.305.1591147318486; Tue, 02 Jun 2020 18:21:58 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1591147318; cv=none; d=google.com; s=arc-20160816; b=vH9uH2zJpTTLHfDaMBNH0pACuTuB2hPbLQdBw0qro1GsFir9jBy8YKfacMj0oENk4e RvOR5I0OCdFrGlNfa7wdedEd7gQ6AmV01kAQ99lcm4v8ZRr+N2g8znETSX6xBLQaX+Dr 40UUQN4zul5QBZxGHHUfjl6NdZ6XJmB2Uf6nQN6LscpjACyWDHnI3MPtlQ2oxSkrvktg Xrf0IcxA+zAXdqY5kZoq0KjQHPP9Yja4xxyh/dZX+4zM+E3megwNb7/jEqdJCUXySjQC c0x4Fa3Zx8wbDqO+ski0rYC9JsW51QOlQUOveNvFTB3kxiboyPOEjoWYrv1lWzscucOl Lm9A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=ZVtlHoEiwSpE9aCTdsiBJjr9uOzV2JRURGAMUjJ7q+Y=; b=YsQKrJMP4rgF/lxLhBNiycDauBrSb2ZI62y/nlV0NQxEtLEsPiPnRIG7PfsCctHTPW FDwJSS46K3FGsMnYXASmuWDTA8cr+rL7moHVf5zLX2Mx7MzwmX3xUguebPAzYo07QioV 3XP0m2KyImmxnvb5qXNpzvKuB3b6wZAorQFBLqPkTKptUjk6Ceg/DZZWQUdeX3qYq/00 0yKm3bPSV/04zgIOjSHH2Fm6ZcbKCeg+tz80QkC07e/HBxRYNOOSd4G+0Jqgg6We9kOZ vEyF6ebHOMQHH3NKhEp285KLcQXF+MuJoMNKfEljzUFzjo/TzdhkkfQmEuO6UGEJqK7s uPTw== ARC-Authentication-Results: i=1; mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=ELVIrvas; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id r140si361498ybc.245.2020.06.02.18.21.58 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 02 Jun 2020 18:21:58 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=ELVIrvas; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:56074 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jgI6P-0002ih-Sy for patch@linaro.org; Tue, 02 Jun 2020 21:21:57 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:48778) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jgHyX-00047V-Se for qemu-devel@nongnu.org; Tue, 02 Jun 2020 21:13:49 -0400 Received: from mail-pl1-x642.google.com ([2607:f8b0:4864:20::642]:42496) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1jgHyT-0003aY-8u for qemu-devel@nongnu.org; Tue, 02 Jun 2020 21:13:49 -0400 Received: by mail-pl1-x642.google.com with SMTP id x11so194278plv.9 for ; Tue, 02 Jun 2020 18:13:44 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=ZVtlHoEiwSpE9aCTdsiBJjr9uOzV2JRURGAMUjJ7q+Y=; b=ELVIrvasf2tXjdwCDv4fGSpi+HULdjLk/umwuLXIsDA/qJVtrIXrjN/b6J04jt+uqo W61mNC5TnHSdB5ZxY6MS1O4QPQ7tu24nYIVE1ao/lRmGQaiYWdRJXF52r/poEPfRjbQj A7WOfIlywtLzegOpYHSjOSiBOTtxF9Yn3p0YDZ4vtW40er396eIySep3/Zk20YIEUj5p fo8U+AvKENBAPY9IRYWqWMi2h64eHYAkFdoYJMhYj811O13zdiiXH4VUOOh2dfMsDrZO sALw5GCPbQWHLXFzDhF6s6ooZdFvGd+/t/Y6ijU+5hK8yR7YOwqPUb5xHmv4Cr2m0EhK WkqQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=ZVtlHoEiwSpE9aCTdsiBJjr9uOzV2JRURGAMUjJ7q+Y=; b=lKlaDjowz4DpCVQ2Oa8jIeLNLOgCFmYgO5QDSEECo2dvBXBg1DzkwYDLdRrN31peKr CZ8SsEUeju8Azaq1JgY8kamQ6nFKCIrWTnvKABSJOh6DsS3Hxldpgisxy9mZ095A7FV1 tnyWqc5fA5h6BE9OGa2beNMdwKNf5biOeCkCU0XRXe6J8u2piirllChbvCBLn4/wnkJD g+XsAlSNSwcmNDf7mweQGE8SdCfSPon7E5JfdCVskNzTySsMaSeygoYZzyY1ttertgjZ zC/dc7PyouLNbOHvoqKIID5vOCDcpRuTXF4Am4XJG+ivIxz94ZDzwzXuxH+4VIGn43f9 up0g== X-Gm-Message-State: AOAM532OAtvd1iezQpt3K0av3a5cECDFS8glZpZdorj10vn1j53rPf3n B9xrAlZk2870JVgHEPBWFzu9RAqvd28= X-Received: by 2002:a17:902:bcc1:: with SMTP id o1mr19151784pls.324.1591146823380; Tue, 02 Jun 2020 18:13:43 -0700 (PDT) Received: from localhost.localdomain (174-21-143-238.tukw.qwest.net. [174.21.143.238]) by smtp.gmail.com with ESMTPSA id 3sm290067pfe.85.2020.06.02.18.13.42 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 02 Jun 2020 18:13:42 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v7 19/42] target/arm: Implement the LDGM, STGM, STZGM instructions Date: Tue, 2 Jun 2020 18:12:54 -0700 Message-Id: <20200603011317.473934-20-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200603011317.473934-1-richard.henderson@linaro.org> References: <20200603011317.473934-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::642; envelope-from=richard.henderson@linaro.org; helo=mail-pl1-x642.google.com X-detected-operating-system: by eggs.gnu.org: No matching host in p0f cache. That's all we know. X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: peter.maydell@linaro.org, qemu-arm@nongnu.org, steplong@quicinc.com Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" Signed-off-by: Richard Henderson --- v3: Require pre-cleaned addresses. v6: Check full mte enabled. Reorg the helpers. --- target/arm/helper-a64.h | 3 ++ target/arm/translate.h | 2 + target/arm/mte_helper.c | 84 ++++++++++++++++++++++++++++++++++++++ target/arm/translate-a64.c | 74 +++++++++++++++++++++++++++++---- 4 files changed, 154 insertions(+), 9 deletions(-) -- 2.25.1 Reviewed-by: Peter Maydell diff --git a/target/arm/helper-a64.h b/target/arm/helper-a64.h index 2fa61b86fa..7b628d100e 100644 --- a/target/arm/helper-a64.h +++ b/target/arm/helper-a64.h @@ -113,3 +113,6 @@ DEF_HELPER_FLAGS_2(stg_stub, TCG_CALL_NO_WG, void, env, i64) DEF_HELPER_FLAGS_3(st2g, TCG_CALL_NO_WG, void, env, i64, i64) DEF_HELPER_FLAGS_3(st2g_parallel, TCG_CALL_NO_WG, void, env, i64, i64) DEF_HELPER_FLAGS_2(st2g_stub, TCG_CALL_NO_WG, void, env, i64) +DEF_HELPER_FLAGS_2(ldgm, TCG_CALL_NO_WG, i64, env, i64) +DEF_HELPER_FLAGS_3(stgm, TCG_CALL_NO_WG, void, env, i64, i64) +DEF_HELPER_FLAGS_3(stzgm_tags, TCG_CALL_NO_WG, void, env, i64, i64) diff --git a/target/arm/translate.h b/target/arm/translate.h index 4fb0feece9..5f51c334a0 100644 --- a/target/arm/translate.h +++ b/target/arm/translate.h @@ -91,6 +91,8 @@ typedef struct DisasContext { * < 0, set by the current instruction. */ int8_t btype; + /* A copy of cpu->dcz_blocksize. */ + uint8_t dcz_blocksize; /* True if this page is guarded. */ bool guarded_page; /* Bottom two bits of XScale c15_cpar coprocessor access control reg */ diff --git a/target/arm/mte_helper.c b/target/arm/mte_helper.c index 7ec7930dfc..27d4b4536c 100644 --- a/target/arm/mte_helper.c +++ b/target/arm/mte_helper.c @@ -274,3 +274,87 @@ void HELPER(st2g_stub)(CPUARMState *env, uint64_t ptr) probe_write(env, ptr + TAG_GRANULE, TAG_GRANULE, mmu_idx, ra); } } + +#define LDGM_STGM_SIZE (4 << GMID_EL1_BS) + +uint64_t HELPER(ldgm)(CPUARMState *env, uint64_t ptr) +{ + int mmu_idx = cpu_mmu_index(env, false); + uintptr_t ra = GETPC(); + void *tag_mem; + + ptr = QEMU_ALIGN_DOWN(ptr, LDGM_STGM_SIZE); + + /* Trap if accessing an invalid page. */ + tag_mem = allocation_tag_mem(env, mmu_idx, ptr, MMU_DATA_LOAD, + LDGM_STGM_SIZE, MMU_DATA_LOAD, + LDGM_STGM_SIZE / (2 * TAG_GRANULE), ra); + + /* The tag is squashed to zero if the page does not support tags. */ + if (!tag_mem) { + return 0; + } + + QEMU_BUILD_BUG_ON(GMID_EL1_BS != 6); + /* + * We are loading 64-bits worth of tags. The ordering of elements + * within the word corresponds to a 64-bit little-endian operation. + */ + return ldq_le_p(tag_mem); +} + +void HELPER(stgm)(CPUARMState *env, uint64_t ptr, uint64_t val) +{ + int mmu_idx = cpu_mmu_index(env, false); + uintptr_t ra = GETPC(); + void *tag_mem; + + ptr = QEMU_ALIGN_DOWN(ptr, LDGM_STGM_SIZE); + + /* Trap if accessing an invalid page. */ + tag_mem = allocation_tag_mem(env, mmu_idx, ptr, MMU_DATA_STORE, + LDGM_STGM_SIZE, MMU_DATA_LOAD, + LDGM_STGM_SIZE / (2 * TAG_GRANULE), ra); + + /* + * Tag store only happens if the page support tags, + * and if the OS has enabled access to the tags. + */ + if (!tag_mem) { + return; + } + + QEMU_BUILD_BUG_ON(GMID_EL1_BS != 6); + /* + * We are storing 64-bits worth of tags. The ordering of elements + * within the word corresponds to a 64-bit little-endian operation. + */ + stq_le_p(tag_mem, val); +} + +void HELPER(stzgm_tags)(CPUARMState *env, uint64_t ptr, uint64_t val) +{ + uintptr_t ra = GETPC(); + int mmu_idx = cpu_mmu_index(env, false); + int log2_dcz_bytes, log2_tag_bytes; + intptr_t dcz_bytes, tag_bytes; + uint8_t *mem; + + /* + * In arm_cpu_realizefn, we assert that dcz > LOG2_TAG_GRANULE+1, + * i.e. 32 bytes, which is an unreasonably small dcz anyway, + * to make sure that we can access one complete tag byte here. + */ + log2_dcz_bytes = env_archcpu(env)->dcz_blocksize + 2; + log2_tag_bytes = log2_dcz_bytes - (LOG2_TAG_GRANULE + 1); + dcz_bytes = (intptr_t)1 << log2_dcz_bytes; + tag_bytes = (intptr_t)1 << log2_tag_bytes; + ptr &= -dcz_bytes; + + mem = allocation_tag_mem(env, mmu_idx, ptr, MMU_DATA_STORE, dcz_bytes, + MMU_DATA_STORE, tag_bytes, ra); + if (mem) { + int tag_pair = (val & 0xf) * 0x11; + memset(mem, tag_pair, tag_bytes); + } +} diff --git a/target/arm/translate-a64.c b/target/arm/translate-a64.c index 2fc070ab90..c4f86c69ba 100644 --- a/target/arm/translate-a64.c +++ b/target/arm/translate-a64.c @@ -3727,7 +3727,7 @@ static void disas_ldst_tag(DisasContext *s, uint32_t insn) uint64_t offset = sextract64(insn, 12, 9) << LOG2_TAG_GRANULE; int op2 = extract32(insn, 10, 2); int op1 = extract32(insn, 22, 2); - bool is_load = false, is_pair = false, is_zero = false; + bool is_load = false, is_pair = false, is_zero = false, is_mult = false; int index = 0; TCGv_i64 addr, clean_addr, tcg_rt; @@ -3743,13 +3743,18 @@ static void disas_ldst_tag(DisasContext *s, uint32_t insn) * > 0 : pre-index, writeback */ switch (op1) { - case 0: /* STG */ + case 0: if (op2 != 0) { /* STG */ index = op2 - 2; - break; + } else { + /* STZGM */ + if (s->current_el == 0 || offset != 0) { + goto do_unallocated; + } + is_mult = is_zero = true; } - goto do_unallocated; + break; case 1: if (op2 != 0) { /* STZG */ @@ -3765,17 +3770,27 @@ static void disas_ldst_tag(DisasContext *s, uint32_t insn) /* ST2G */ is_pair = true; index = op2 - 2; - break; + } else { + /* STGM */ + if (s->current_el == 0 || offset != 0) { + goto do_unallocated; + } + is_mult = true; } - goto do_unallocated; + break; case 3: if (op2 != 0) { /* STZ2G */ is_pair = is_zero = true; index = op2 - 2; - break; + } else { + /* LDGM */ + if (s->current_el == 0 || offset != 0) { + goto do_unallocated; + } + is_mult = is_load = true; } - goto do_unallocated; + break; default: do_unallocated: @@ -3783,7 +3798,9 @@ static void disas_ldst_tag(DisasContext *s, uint32_t insn) return; } - if (!dc_isar_feature(aa64_mte_insn_reg, s)) { + if (is_mult + ? !dc_isar_feature(aa64_mte, s) + : !dc_isar_feature(aa64_mte_insn_reg, s)) { goto do_unallocated; } @@ -3797,6 +3814,44 @@ static void disas_ldst_tag(DisasContext *s, uint32_t insn) tcg_gen_addi_i64(addr, addr, offset); } + if (is_mult) { + tcg_rt = cpu_reg(s, rt); + + if (is_zero) { + int size = 4 << s->dcz_blocksize; + + if (s->ata) { + gen_helper_stzgm_tags(cpu_env, addr, tcg_rt); + } + /* + * The non-tags portion of STZGM is mostly like DC_ZVA, + * except the alignment happens before the access. + */ + clean_addr = clean_data_tbi(s, addr); + tcg_gen_andi_i64(clean_addr, clean_addr, -size); + gen_helper_dc_zva(cpu_env, clean_addr); + } else if (s->ata) { + if (is_load) { + gen_helper_ldgm(tcg_rt, cpu_env, addr); + } else { + gen_helper_stgm(cpu_env, addr, tcg_rt); + } + } else { + MMUAccessType acc = is_load ? MMU_DATA_LOAD : MMU_DATA_STORE; + int size = 4 << GMID_EL1_BS; + + clean_addr = clean_data_tbi(s, addr); + tcg_gen_andi_i64(clean_addr, clean_addr, -size); + gen_probe_access(s, clean_addr, acc, size); + + if (is_load) { + /* The result tags are zeros. */ + tcg_gen_movi_i64(tcg_rt, 0); + } + } + return; + } + if (is_load) { tcg_rt = cpu_reg(s, rt); if (s->ata) { @@ -14501,6 +14556,7 @@ static void aarch64_tr_init_disas_context(DisasContextBase *dcbase, dc->vec_stride = 0; dc->cp_regs = arm_cpu->cp_regs; dc->features = env->features; + dc->dcz_blocksize = arm_cpu->dcz_blocksize; /* Single step state. The code-generation logic here is: * SS_ACTIVE == 0: From patchwork Wed Jun 3 01:12:55 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 187421 Delivered-To: patch@linaro.org Received: by 2002:a92:cf06:0:0:0:0:0 with SMTP id c6csp75438ilo; Tue, 2 Jun 2020 18:23:36 -0700 (PDT) X-Google-Smtp-Source: ABdhPJw+vepQh1E+inpp5OTimMzG8pnV9JVCW1kALuoYaUuPmVodEL7EzWf6O5QmCHETGg8sKRgF X-Received: by 2002:a25:7610:: with SMTP id r16mr4210208ybc.75.1591147416160; Tue, 02 Jun 2020 18:23:36 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1591147416; cv=none; d=google.com; s=arc-20160816; b=ZVxNYIt2qGLAYEC5O3upsyangbWT20vSDNI0zipvpAcgR3lxCs7JKJEoejNYRZ8stj TpFWoqK6q/BpTvKb3t4T3zlbrBI8YwLpB8a+2MZhUVOlJZNoOZ62rKfBGjtUgh737Yot hh0Fu0IymD+Kt04KM/ay0Xbn0u3MR09kljMJpHdRN9KIXmrNXP4taSQcxtS/lQKJfHte tu1u71X/YtXlk95uI1L9SjgOc8T5eWM/RbnpfJfs/dV+DFq2ogSmYGTX6yuIYzuO5UNv JK4yuF2bpzOHsa7DsluAO2Cr9pJCgjGtNFgU1w5LOcteSjgExibM7BcxZuxTko7x4MGs 5w8Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=cf68FCAzJTbpJLd19R46t9jMSqR5E/27UO36TFsKmfE=; b=H5wgaf8o1mb/o4Bh8d7W088bFYgG6H6cxtSNMd8Z8tmwkg7YTXnqgT5s8uAT9GFUJE qOy9v76qbFYQpY/g2kheBsd6qB3w3TzyNai95ig4CrlxPf6I3DUkGbW19urWYai+Y/RG Sb/eKyQPQ1FsN+mlhsWbfun5vXBhRD3J0zPcFie+GimfA6As7XLgQyohf4fdnJkavuKx dqRgeZKHjacgdXuotLXuMxsD3NY6CQE2doHwHzd+dsE7FwEUnH7Shw7IXibE9PROt39G s/qZsBgBEjgBHVMRPkzVRJYtIp6RirV90YT3QwxyBd2kQTg23kNbs3P0T1yIxssveidH YRCg== ARC-Authentication-Results: i=1; mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=DXIha4Y+; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id x7si325386ybm.67.2020.06.02.18.23.36 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 02 Jun 2020 18:23:36 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=DXIha4Y+; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:37540 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jgI7z-0006gn-Ib for patch@linaro.org; Tue, 02 Jun 2020 21:23:35 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:48818) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jgHya-0004G1-Qm for qemu-devel@nongnu.org; Tue, 02 Jun 2020 21:13:52 -0400 Received: from mail-pf1-x444.google.com ([2607:f8b0:4864:20::444]:45168) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1jgHyU-0003aq-VI for qemu-devel@nongnu.org; Tue, 02 Jun 2020 21:13:52 -0400 Received: by mail-pf1-x444.google.com with SMTP id a127so397775pfa.12 for ; Tue, 02 Jun 2020 18:13:46 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=cf68FCAzJTbpJLd19R46t9jMSqR5E/27UO36TFsKmfE=; b=DXIha4Y+kZZ0jwtry5maEHtc4leXvNE/8F5mrjkukxbgt17m5nWPgXa0F3cO6mjEQF i/GT53cj97p4ydXdKPosc+DegwTAhdpBayLuUYcE9FvGQWK53P2BF4sHo7WlLsBxMwxn 0GAwTzyJHdRaYC8BsBoWyaGpQVY0siYATZRycZshKpBJcM08iajnkPLMBF5Aim8yZdO0 n1lnGBLTAxu28tosEuMu8QTMpdyIXr+oCGkm8b1s10q8zbv2m+nIS3EdtkRzqfUXxolP OBJakcVXO2XGbueCmdHDHFs9+8s3GPTwn8BWjwN1zIMptBwf8LlUaln6vr3AtielOxjS 5iPg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=cf68FCAzJTbpJLd19R46t9jMSqR5E/27UO36TFsKmfE=; b=aGH/dXvZA0Fycu9s0cV3PsS5Cf0uHa0jj/vYcIBngJnwMfAUoBobnMAHHhUMwLGvk9 CeviOQg9GT9xCFd8JsCcWE/4uWpuYII5LcVslTc+VN4Unewnoze2CpHMBKMHm9KRiqw4 lxRn+unkWlhM80guFSed10c+TEvupMA29qp8NH4qGd+YXuVRKDI/8AUap8Q+zkn9/kvN 28Xp7Oui17Gt8xi/rVpkXxCeXblNmFQoLOmI6BDuFVh73UCfe95j0ONf4xd6yiJLHj1e jZO/pMd9xpfzAB0QJ1sjUI3S5PRs4pPcZ+I4EaYZ5i+SREEkePmq8t1xwlp5uP/VyKGW n+2g== X-Gm-Message-State: AOAM531RZWH9Fz6P7zAY55FSQHsKQtdyZVHNa0kumm3aoKSJs1g+jK6y VXkHIfXtJBxjREWard3XGyQQmpohAQU= X-Received: by 2002:a17:90a:dd43:: with SMTP id u3mr2120129pjv.221.1591146824770; Tue, 02 Jun 2020 18:13:44 -0700 (PDT) Received: from localhost.localdomain (174-21-143-238.tukw.qwest.net. [174.21.143.238]) by smtp.gmail.com with ESMTPSA id 3sm290067pfe.85.2020.06.02.18.13.43 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 02 Jun 2020 18:13:43 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v7 20/42] target/arm: Implement the access tag cache flushes Date: Tue, 2 Jun 2020 18:12:55 -0700 Message-Id: <20200603011317.473934-21-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200603011317.473934-1-richard.henderson@linaro.org> References: <20200603011317.473934-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::444; envelope-from=richard.henderson@linaro.org; helo=mail-pf1-x444.google.com X-detected-operating-system: by eggs.gnu.org: No matching host in p0f cache. That's all we know. X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: peter.maydell@linaro.org, qemu-arm@nongnu.org, steplong@quicinc.com Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" Like the regular data cache flushes, these are nops within qemu. Signed-off-by: Richard Henderson --- v6: Split out and handle el0 cache ops properly. --- target/arm/helper.c | 65 +++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 65 insertions(+) -- 2.25.1 Reviewed-by: Peter Maydell diff --git a/target/arm/helper.c b/target/arm/helper.c index 8aaa6f22b2..20d7bf4ee1 100644 --- a/target/arm/helper.c +++ b/target/arm/helper.c @@ -6927,6 +6927,32 @@ static const ARMCPRegInfo mte_reginfo[] = { .opc0 = 3, .opc1 = 3, .crn = 4, .crm = 2, .opc2 = 7, .type = ARM_CP_NO_RAW, .access = PL0_RW, .readfn = tco_read, .writefn = tco_write }, + { .name = "DC_IGVAC", .state = ARM_CP_STATE_AA64, + .opc0 = 1, .opc1 = 0, .crn = 7, .crm = 6, .opc2 = 3, + .type = ARM_CP_NOP, .access = PL1_W, + .accessfn = aa64_cacheop_poc_access }, + { .name = "DC_IGSW", .state = ARM_CP_STATE_AA64, + .opc0 = 1, .opc1 = 0, .crn = 7, .crm = 6, .opc2 = 4, + .type = ARM_CP_NOP, .access = PL1_W, .accessfn = access_tsw }, + { .name = "DC_IGDVAC", .state = ARM_CP_STATE_AA64, + .opc0 = 1, .opc1 = 0, .crn = 7, .crm = 6, .opc2 = 5, + .type = ARM_CP_NOP, .access = PL1_W, + .accessfn = aa64_cacheop_poc_access }, + { .name = "DC_IGDSW", .state = ARM_CP_STATE_AA64, + .opc0 = 1, .opc1 = 0, .crn = 7, .crm = 6, .opc2 = 6, + .type = ARM_CP_NOP, .access = PL1_W, .accessfn = access_tsw }, + { .name = "DC_CGSW", .state = ARM_CP_STATE_AA64, + .opc0 = 1, .opc1 = 0, .crn = 7, .crm = 10, .opc2 = 4, + .type = ARM_CP_NOP, .access = PL1_W, .accessfn = access_tsw }, + { .name = "DC_CGDSW", .state = ARM_CP_STATE_AA64, + .opc0 = 1, .opc1 = 0, .crn = 7, .crm = 10, .opc2 = 6, + .type = ARM_CP_NOP, .access = PL1_W, .accessfn = access_tsw }, + { .name = "DC_CIGSW", .state = ARM_CP_STATE_AA64, + .opc0 = 1, .opc1 = 0, .crn = 7, .crm = 14, .opc2 = 4, + .type = ARM_CP_NOP, .access = PL1_W, .accessfn = access_tsw }, + { .name = "DC_CIGDSW", .state = ARM_CP_STATE_AA64, + .opc0 = 1, .opc1 = 0, .crn = 7, .crm = 14, .opc2 = 6, + .type = ARM_CP_NOP, .access = PL1_W, .accessfn = access_tsw }, REGINFO_SENTINEL }; @@ -6936,6 +6962,43 @@ static const ARMCPRegInfo mte_tco_ro_reginfo[] = { .type = ARM_CP_CONST, .access = PL0_RW, }, REGINFO_SENTINEL }; + +static const ARMCPRegInfo mte_el0_cacheop_reginfo[] = { + { .name = "DC_CGVAC", .state = ARM_CP_STATE_AA64, + .opc0 = 1, .opc1 = 3, .crn = 7, .crm = 10, .opc2 = 3, + .type = ARM_CP_NOP, .access = PL0_W, + .accessfn = aa64_cacheop_poc_access }, + { .name = "DC_CGDVAC", .state = ARM_CP_STATE_AA64, + .opc0 = 1, .opc1 = 3, .crn = 7, .crm = 10, .opc2 = 5, + .type = ARM_CP_NOP, .access = PL0_W, + .accessfn = aa64_cacheop_poc_access }, + { .name = "DC_CGVAP", .state = ARM_CP_STATE_AA64, + .opc0 = 1, .opc1 = 3, .crn = 7, .crm = 12, .opc2 = 3, + .type = ARM_CP_NOP, .access = PL0_W, + .accessfn = aa64_cacheop_poc_access }, + { .name = "DC_CGDVAP", .state = ARM_CP_STATE_AA64, + .opc0 = 1, .opc1 = 3, .crn = 7, .crm = 12, .opc2 = 5, + .type = ARM_CP_NOP, .access = PL0_W, + .accessfn = aa64_cacheop_poc_access }, + { .name = "DC_CGVADP", .state = ARM_CP_STATE_AA64, + .opc0 = 1, .opc1 = 3, .crn = 7, .crm = 13, .opc2 = 3, + .type = ARM_CP_NOP, .access = PL0_W, + .accessfn = aa64_cacheop_poc_access }, + { .name = "DC_CGDVADP", .state = ARM_CP_STATE_AA64, + .opc0 = 1, .opc1 = 3, .crn = 7, .crm = 13, .opc2 = 5, + .type = ARM_CP_NOP, .access = PL0_W, + .accessfn = aa64_cacheop_poc_access }, + { .name = "DC_CIGVAC", .state = ARM_CP_STATE_AA64, + .opc0 = 1, .opc1 = 3, .crn = 7, .crm = 14, .opc2 = 3, + .type = ARM_CP_NOP, .access = PL0_W, + .accessfn = aa64_cacheop_poc_access }, + { .name = "DC_CIGDVAC", .state = ARM_CP_STATE_AA64, + .opc0 = 1, .opc1 = 3, .crn = 7, .crm = 14, .opc2 = 5, + .type = ARM_CP_NOP, .access = PL0_W, + .accessfn = aa64_cacheop_poc_access }, + REGINFO_SENTINEL +}; + #endif static CPAccessResult access_predinv(CPUARMState *env, const ARMCPRegInfo *ri, @@ -8069,8 +8132,10 @@ void register_cp_regs_for_features(ARMCPU *cpu) */ if (cpu_isar_feature(aa64_mte, cpu)) { define_arm_cp_regs(cpu, mte_reginfo); + define_arm_cp_regs(cpu, mte_el0_cacheop_reginfo); } else if (cpu_isar_feature(aa64_mte_insn_reg, cpu)) { define_arm_cp_regs(cpu, mte_tco_ro_reginfo); + define_arm_cp_regs(cpu, mte_el0_cacheop_reginfo); } #endif From patchwork Wed Jun 3 01:12:56 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 187426 Delivered-To: patch@linaro.org Received: by 2002:a92:cf06:0:0:0:0:0 with SMTP id c6csp76925ilo; Tue, 2 Jun 2020 18:26:20 -0700 (PDT) X-Google-Smtp-Source: ABdhPJzS9Mm9yhmkQjpzqWi3nHqSIiqwupurywuf9v540AtgHmaGcdV2Or7x5xNbyB3n4WDbx2T2 X-Received: by 2002:a25:c7cd:: with SMTP id w196mr44597399ybe.482.1591147580236; Tue, 02 Jun 2020 18:26:20 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1591147580; cv=none; d=google.com; s=arc-20160816; b=njD9eC81kBRUR3auiKcpeQOxL+Bd7jWGEY7powpeb/D7qgUsw59IcMhJHYAwmdqfm3 X6s9zEjHNc6MWQBk0b3fqTMW52ER6mPgJVr3GAPf3quLBdI/7x5BeNanZcDtTXzssjKA sznT0VKR2kQctjUpVDo/rDoyOwInDzPKLNAyYv+5FE+t5bXZl6ZsVwa25aukqqVLvG7I MV8Xh1oNIBi2KsfI+VF9q3GZaY63C5cZx+MYY2HLT9pIHY7ffHaLOd0TqygKUxAjp4aX tgiXy0CZzDrNdXVyebK3XzFLhlEGTkjrqx9bQGmqTUWaxiWc5Pkw2HsmyLTZ605wRMZp 4rYg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=AEdNLYFCyfyMqsP9U02w6IsvCKyLZbP9JA6vNA4o+yc=; b=wZ9D5/sqwI4dBpjc6cD9DO4ZU6MbfgEu8mc9CDcQzsJLVzFlX1YVrqiQD8yD3mJiUP Nu5FTRVi5vGseRlpsf6OSSJUixFX4bo3sDO/YY75zUWnCP6lbX5usdpIjGt+9IeDtJI1 6oxSWTh4CMdjxVCnoGGOKYDNuDrfcKGoScCtCyt/V8qMN9ENvvBxy1AU1/0bEL/ykGYP J20Lym5zHC7knLzWn3eO5vNs9ccJJqSefydMN6Uu/zbZd+DvP2YJ1x8+I2F3vQYYHZVG TMqxYWXeHH7SfeuCqchULuYeEzCG5ahvMf5ng/7HWj0z5yxh1yeW0RAdVLdUmP0KigbQ FBwQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=PGfxWPg8; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id d3si288458ybm.305.2020.06.02.18.26.20 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 02 Jun 2020 18:26:20 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=PGfxWPg8; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:47938 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jgIAd-0002Wk-LV for patch@linaro.org; Tue, 02 Jun 2020 21:26:19 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:48792) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jgHyZ-0004Ao-1v for qemu-devel@nongnu.org; Tue, 02 Jun 2020 21:13:51 -0400 Received: from mail-pl1-x631.google.com ([2607:f8b0:4864:20::631]:45652) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1jgHyV-0003az-TE for qemu-devel@nongnu.org; Tue, 02 Jun 2020 21:13:50 -0400 Received: by mail-pl1-x631.google.com with SMTP id y11so187214plt.12 for ; Tue, 02 Jun 2020 18:13:47 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=AEdNLYFCyfyMqsP9U02w6IsvCKyLZbP9JA6vNA4o+yc=; b=PGfxWPg8A1ZHVs+Pwb/SugE/9/Q8U0LxulnXFVjawH4hRZaDumiiZn3ZUdyua63jpg n57CX7aHu0LX4mzYqCACULh4KfDwfF7kKGZgiISeaviByx2mjRpvsMUTuUnk0IDV1oF1 A/Hjays5tdQMQ4ZN8GS8CVbC88fRc84tgaUxHzRlj/mJlKeudMATl3TKQMxe5QX7TDJd zIPh55y5e0ttT7MrLE+sJUpvu+J3p3sOlwx0UUKad21fZ68OwOLvq+n61ndyMrDttBSE uQPC37xd9nR1ZzBGraq3Nd3CDbYR31hqdOnkgSJdq0ciY/x3MwVxEsjPXMpkcReGIOFu c8Hw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=AEdNLYFCyfyMqsP9U02w6IsvCKyLZbP9JA6vNA4o+yc=; b=CBrIZBl213Yl57zIcRwxRlYFD7JckJceMjQKx8oyjfIpL/zJHm7X9dJAXydkQJYDI4 3Bxnr5JKL47EuNu919WKh57h/iGjuyXuMP2lltDBwesj1DhzfPFU5MbOzWLIUREYenLe dDnqQs7tB73FC/KyQpHCn8b1TGX1QmhL3ebiEYTUdedbpH19tit+nCOjA/QEVm0eaxhA UHL4hY9ahpTvBx8nwHwJfy/Zio4SDu56tbe8za4hc4K9Hv4bFUrGtw9O56zOqvnshkl6 +lxROpW0iJbrWxkdwwASqrcQkzfjPUvvzYLqt2ZlQqvLMb9h/h1nkA4HOe/2Hl1E0L1y RuOg== X-Gm-Message-State: AOAM533MrtBVD038eWkwg+AbzLUiYzeUBFrJj6z9RVCi4lI0foeFCxe8 hs0QYp1qmXVsTgLMG89VrztdHby+ZMk= X-Received: by 2002:a17:902:694b:: with SMTP id k11mr25067287plt.77.1591146825964; Tue, 02 Jun 2020 18:13:45 -0700 (PDT) Received: from localhost.localdomain (174-21-143-238.tukw.qwest.net. [174.21.143.238]) by smtp.gmail.com with ESMTPSA id 3sm290067pfe.85.2020.06.02.18.13.44 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 02 Jun 2020 18:13:45 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v7 21/42] target/arm: Move regime_el to internals.h Date: Tue, 2 Jun 2020 18:12:56 -0700 Message-Id: <20200603011317.473934-22-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200603011317.473934-1-richard.henderson@linaro.org> References: <20200603011317.473934-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::631; envelope-from=richard.henderson@linaro.org; helo=mail-pl1-x631.google.com X-detected-operating-system: by eggs.gnu.org: No matching host in p0f cache. That's all we know. X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: peter.maydell@linaro.org, qemu-arm@nongnu.org, steplong@quicinc.com Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" We will shortly need this in mte_helper.c as well. Signed-off-by: Richard Henderson --- target/arm/internals.h | 36 ++++++++++++++++++++++++++++++++++++ target/arm/helper.c | 36 ------------------------------------ 2 files changed, 36 insertions(+), 36 deletions(-) -- 2.25.1 Reviewed-by: Peter Maydell diff --git a/target/arm/internals.h b/target/arm/internals.h index 5c69d4e5a5..c36fcb151b 100644 --- a/target/arm/internals.h +++ b/target/arm/internals.h @@ -913,6 +913,42 @@ static inline bool regime_is_pan(CPUARMState *env, ARMMMUIdx mmu_idx) } } +/* Return the exception level which controls this address translation regime */ +static inline uint32_t regime_el(CPUARMState *env, ARMMMUIdx mmu_idx) +{ + switch (mmu_idx) { + case ARMMMUIdx_E20_0: + case ARMMMUIdx_E20_2: + case ARMMMUIdx_E20_2_PAN: + case ARMMMUIdx_Stage2: + case ARMMMUIdx_E2: + return 2; + case ARMMMUIdx_SE3: + return 3; + case ARMMMUIdx_SE10_0: + return arm_el_is_aa64(env, 3) ? 1 : 3; + case ARMMMUIdx_SE10_1: + case ARMMMUIdx_SE10_1_PAN: + case ARMMMUIdx_Stage1_E0: + case ARMMMUIdx_Stage1_E1: + case ARMMMUIdx_Stage1_E1_PAN: + case ARMMMUIdx_E10_0: + case ARMMMUIdx_E10_1: + case ARMMMUIdx_E10_1_PAN: + case ARMMMUIdx_MPrivNegPri: + case ARMMMUIdx_MUserNegPri: + case ARMMMUIdx_MPriv: + case ARMMMUIdx_MUser: + case ARMMMUIdx_MSPrivNegPri: + case ARMMMUIdx_MSUserNegPri: + case ARMMMUIdx_MSPriv: + case ARMMMUIdx_MSUser: + return 1; + default: + g_assert_not_reached(); + } +} + /* Return the FSR value for a debug exception (watchpoint, hardware * breakpoint or BKPT insn) targeting the specified exception level. */ diff --git a/target/arm/helper.c b/target/arm/helper.c index 20d7bf4ee1..bad639e2a7 100644 --- a/target/arm/helper.c +++ b/target/arm/helper.c @@ -9791,42 +9791,6 @@ void arm_cpu_do_interrupt(CPUState *cs) } #endif /* !CONFIG_USER_ONLY */ -/* Return the exception level which controls this address translation regime */ -static uint32_t regime_el(CPUARMState *env, ARMMMUIdx mmu_idx) -{ - switch (mmu_idx) { - case ARMMMUIdx_E20_0: - case ARMMMUIdx_E20_2: - case ARMMMUIdx_E20_2_PAN: - case ARMMMUIdx_Stage2: - case ARMMMUIdx_E2: - return 2; - case ARMMMUIdx_SE3: - return 3; - case ARMMMUIdx_SE10_0: - return arm_el_is_aa64(env, 3) ? 1 : 3; - case ARMMMUIdx_SE10_1: - case ARMMMUIdx_SE10_1_PAN: - case ARMMMUIdx_Stage1_E0: - case ARMMMUIdx_Stage1_E1: - case ARMMMUIdx_Stage1_E1_PAN: - case ARMMMUIdx_E10_0: - case ARMMMUIdx_E10_1: - case ARMMMUIdx_E10_1_PAN: - case ARMMMUIdx_MPrivNegPri: - case ARMMMUIdx_MUserNegPri: - case ARMMMUIdx_MPriv: - case ARMMMUIdx_MUser: - case ARMMMUIdx_MSPrivNegPri: - case ARMMMUIdx_MSUserNegPri: - case ARMMMUIdx_MSPriv: - case ARMMMUIdx_MSUser: - return 1; - default: - g_assert_not_reached(); - } -} - uint64_t arm_sctlr(CPUARMState *env, int el) { /* Only EL0 needs to be adjusted for EL1&0 or EL2&0. */ From patchwork Wed Jun 3 01:12:57 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 187431 Delivered-To: patch@linaro.org Received: by 2002:a92:cf06:0:0:0:0:0 with SMTP id c6csp78264ilo; Tue, 2 Jun 2020 18:28:43 -0700 (PDT) X-Google-Smtp-Source: ABdhPJycawzaRxIPMiyGsN04Gc8rdAoXz8B6SqElcuzwW7l/xAfWH/1RsuN4pLYygufFHjNkl70s X-Received: by 2002:a25:3489:: with SMTP id b131mr47407323yba.224.1591147723323; Tue, 02 Jun 2020 18:28:43 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1591147723; cv=none; d=google.com; s=arc-20160816; b=Er0kmUz2c1euEAxaBoBN+ZUE1FLPtOmcys88c5BLqVoUd9ug0mZhHL1o+w8vnocGDR ZhQHKPjW31Yc6Pmd+k45QmxGG/iDFs2FXZ3CdueaGAFo/c0kXjFHwg7UXNOKSviMpNrc IFNDIG1ZhXXK8OMHquN8JX2dMZwwmLQljJ0EVQdRnXJOC86wcwf8qLqrcV5jdzWx5FzI NN+WGYYaxADOi4skGmyhrFZGqdjwQLgkhgKltRvO4RByN6C19htLsLn3QaC/FaAs56nG Mb/11LqsdLQRVdEmmWC7lELrF0Af02acdAmkOMjGp3+5gcjaQOwl9q/pznHLUJnaViY+ WU4g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=xQv4B3bb36rbPBYcVwiFiacx7H8VsL7xVtsrr5uJK6w=; b=dmG6xQKv50jN3V31EM/AI7PJQGMoBEfwhE9I603ybz0mvvCF0oWT9hyhoTCpknBUT7 KY7nKaybs1+nOHOv/cAm0/FC24be11UXhHhcu3UcEZiGfIqGiiFU00NW/pGIwcp5sXtE JEZzzmdZQaU0WJXYzoj/XpA/lsa6wIJHYomUO7Xd7tHUs9ikSvtIiQnfRwstfoKSRHIB QKbYQuOxb/B7lvlGhkwMx9LlwZ4XYBULIkNvt4mEI2UucV/KLya+IxnsBr4rV3D5pZnn L2SrQ1wMUEO/C8t4h1YrvAUQ7Qrp1IMC5MTgxr4QzrhqLJ1zJ5VXhSvOhHu5KrPnZ1H5 eveA== ARC-Authentication-Results: i=1; mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=svnfqhtt; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id y7si360496yba.215.2020.06.02.18.28.43 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 02 Jun 2020 18:28:43 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=svnfqhtt; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:57476 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jgICw-0006Ob-QC for patch@linaro.org; Tue, 02 Jun 2020 21:28:42 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:48808) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jgHyZ-0004DE-Pk for qemu-devel@nongnu.org; Tue, 02 Jun 2020 21:13:51 -0400 Received: from mail-pl1-x629.google.com ([2607:f8b0:4864:20::629]:36268) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1jgHyX-0003bV-7H for qemu-devel@nongnu.org; Tue, 02 Jun 2020 21:13:51 -0400 Received: by mail-pl1-x629.google.com with SMTP id bg4so205322plb.3 for ; Tue, 02 Jun 2020 18:13:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=xQv4B3bb36rbPBYcVwiFiacx7H8VsL7xVtsrr5uJK6w=; b=svnfqhttMHOAuUdKTBputqZVA2XoWk84Evd6xO2uOpVuJb39CqHBhz5AGXYlqySWu4 qnP7DQSRNJA7OLbN32jGJMMXUAYuxGBLyXo8S3sJZI1m29I1uAAuzznQHGj4CrOfJzcJ liO22Tw/1WJJhzv3rflsFZaMxSuz2zmGWjcmMdD3Iveq+WywzDAS+m9FblI7CTqtVEIg Qt9oelXng+hVcdEntO1RP1VWHxjJ+9ICLYXjAIk0q/IZCSFI38RaOM+5FoxO53pZh1Al 8cmYKUXldeAXV61oefZN46Ddwf2vmm1d3Oh4sW6ghSft3mBZ1Wygp06lEWyqxKpZZy7+ 9uog== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=xQv4B3bb36rbPBYcVwiFiacx7H8VsL7xVtsrr5uJK6w=; b=HmCdsmw/DVXrRQSbTrLt4y5nLsZ/T2gFoqyUvdrLzEfpJccR/4vXXrMerAP9RfPhYH 937eHzn3ppmGXMGtuAg0qXIYOhYPbquAQPaLaDLmH3N0xAiFeq1HLWzxkY33FapIMej0 nyJ//zhlf2MnN60xeTVlqR+pB5pX6WtCB8Rs3Qy+fNu+0hWSs7AAJ75PD5LMLAhdz45x 0iPrERMzzXnA9X2tcnCdNc3RW2DHU5U6SyHyGlbJXK8g2H6XgErUitdT6XXoKsrxYam0 xKq895HbVVBtwbg26242m/FPGzFA7aF3AFCVyHIrTRdqh0RgwABZRia5TGExjPOVnB/g h2Sg== X-Gm-Message-State: AOAM532HSW+LzZZBcin5U7jwk2prUmlBXLi4sIGkHV0BBX+6jllJ2DB9 Yh3F5o+SvpqnRgpwbwEve0Koo7kjlW0= X-Received: by 2002:a17:902:ee93:: with SMTP id a19mr16191657pld.144.1591146827390; Tue, 02 Jun 2020 18:13:47 -0700 (PDT) Received: from localhost.localdomain (174-21-143-238.tukw.qwest.net. [174.21.143.238]) by smtp.gmail.com with ESMTPSA id 3sm290067pfe.85.2020.06.02.18.13.46 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 02 Jun 2020 18:13:46 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v7 22/42] target/arm: Move regime_tcr to internals.h Date: Tue, 2 Jun 2020 18:12:57 -0700 Message-Id: <20200603011317.473934-23-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200603011317.473934-1-richard.henderson@linaro.org> References: <20200603011317.473934-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::629; envelope-from=richard.henderson@linaro.org; helo=mail-pl1-x629.google.com X-detected-operating-system: by eggs.gnu.org: No matching host in p0f cache. That's all we know. X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: peter.maydell@linaro.org, qemu-arm@nongnu.org, steplong@quicinc.com Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" We will shortly need this in mte_helper.c as well. Signed-off-by: Richard Henderson --- target/arm/internals.h | 9 +++++++++ target/arm/helper.c | 9 --------- 2 files changed, 9 insertions(+), 9 deletions(-) -- 2.25.1 Reviewed-by: Peter Maydell diff --git a/target/arm/internals.h b/target/arm/internals.h index c36fcb151b..7c9abbabc9 100644 --- a/target/arm/internals.h +++ b/target/arm/internals.h @@ -949,6 +949,15 @@ static inline uint32_t regime_el(CPUARMState *env, ARMMMUIdx mmu_idx) } } +/* Return the TCR controlling this translation regime */ +static inline TCR *regime_tcr(CPUARMState *env, ARMMMUIdx mmu_idx) +{ + if (mmu_idx == ARMMMUIdx_Stage2) { + return &env->cp15.vtcr_el2; + } + return &env->cp15.tcr_el[regime_el(env, mmu_idx)]; +} + /* Return the FSR value for a debug exception (watchpoint, hardware * breakpoint or BKPT insn) targeting the specified exception level. */ diff --git a/target/arm/helper.c b/target/arm/helper.c index bad639e2a7..c56e8e500b 100644 --- a/target/arm/helper.c +++ b/target/arm/helper.c @@ -9873,15 +9873,6 @@ static inline uint64_t regime_ttbr(CPUARMState *env, ARMMMUIdx mmu_idx, #endif /* !CONFIG_USER_ONLY */ -/* Return the TCR controlling this translation regime */ -static inline TCR *regime_tcr(CPUARMState *env, ARMMMUIdx mmu_idx) -{ - if (mmu_idx == ARMMMUIdx_Stage2) { - return &env->cp15.vtcr_el2; - } - return &env->cp15.tcr_el[regime_el(env, mmu_idx)]; -} - /* Convert a possible stage1+2 MMU index into the appropriate * stage 1 MMU index */ From patchwork Wed Jun 3 01:12:58 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 187434 Delivered-To: patch@linaro.org Received: by 2002:a92:cf06:0:0:0:0:0 with SMTP id c6csp79347ilo; Tue, 2 Jun 2020 18:30:19 -0700 (PDT) X-Google-Smtp-Source: ABdhPJzzrEC4Ufa3jMcxEUtEgj0CFW3oO3bXiXshp5PUabO/8jltaeH7HagGk49dkLUQnSPzrvAg X-Received: by 2002:a25:b8cb:: with SMTP id g11mr43183121ybm.189.1591147819783; Tue, 02 Jun 2020 18:30:19 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1591147819; cv=none; d=google.com; s=arc-20160816; b=lH1QtJceOUCkS6dYRBiSHf3ZWgtUmDMxRYXWo+J2SCjpUvgDijghV8PNO1iGx1c0AQ pqecskXgJZUT11/71Iun8PF7p6DnvT7C9PRgbzgw4NFKzbNRUf7QorMLNPAIsaBiOi5s UP+ew8k5rSxGHAUeWcXUx1+OIXfN87UQfRyRR9ZAdKTAwdA96JgjipuW4whUV5TGHioZ zNsF5T3VLpfM75zz6vUw7aTtQ5fiWrPhQrlap8QRS/x620YBrOaKs91XUG/+wy3YGVI7 9n754h5PXm2OwO+I2Bx5FG0kAPQ1mYZNMnDia3xFndv0ide19Ex+IaHL3wBgGHAQJ9q9 KQRg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=9ps7TuAQURpuLzr3Fia73pRGmFO7lqbm8nTlS3QkaOw=; b=XT0QsGWfypHL6UOat1Uc3t2YwfNb7yab1jXC7Wmvv6bk8xtChv+C6QSmwTjYWFuCxg AVmGLvSE2e37Dk3axhTbk/odRB/CAumKJwmqZdmKPcvEOsI2Ia43xZJgThPpcyl3YUsi e/2PQ5kwcSQM9fGHwreKb35dFsgQ1CRLDZW4/rTUVmkYL0lqTD/z1WQJH8C0O0F9Fbsh UQ/+LTwk5lJyVsE5DwQ9qoOSwvzsAVowqEBnznnHzThG5iiGQXNMimhczwDBaUCF7DlT q/Ci/wH/4jbjgFbPzkTdJROCC1VhHR4iDE1XcUpYjRfp8lqYYrd3/wLqXYwCiBBo1kLh 0l9g== ARC-Authentication-Results: i=1; mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=eZsa4cZm; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id i2si339282ybg.40.2020.06.02.18.30.19 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 02 Jun 2020 18:30:19 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=eZsa4cZm; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:41152 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jgIEV-0002kx-85 for patch@linaro.org; Tue, 02 Jun 2020 21:30:19 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:48834) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jgHyc-0004Jy-4i for qemu-devel@nongnu.org; Tue, 02 Jun 2020 21:13:54 -0400 Received: from mail-pl1-x643.google.com ([2607:f8b0:4864:20::643]:43511) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1jgHyZ-0003cI-1K for qemu-devel@nongnu.org; Tue, 02 Jun 2020 21:13:53 -0400 Received: by mail-pl1-x643.google.com with SMTP id g12so191413pll.10 for ; Tue, 02 Jun 2020 18:13:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=9ps7TuAQURpuLzr3Fia73pRGmFO7lqbm8nTlS3QkaOw=; b=eZsa4cZm5sTuZ9yP7lA24yfAQQ/XzNMCj+AFdGmcUf2HnJGvUHHk82A6+7vy23gpbC SJVxy4rNidE4c+3ZUVBF4L5Kz7mM8YFx7n9Amy1x6aeIiO39eTQtz7KiPlY87Nh6lo+5 TSS7Uo2T0nJtVzbbrKwYX5hA+JCxWqIZthWpz1I5OhbGcJQdGvnIuM5x3HBRTBWvhBjJ y3Ng/cY+1r9VZwPZ3V5RO9B7k7OUS9HU2FU4b8ZzLcYYXppnFdHm615ZvITLFV8Xkbye 3cA42scxUurmYQD4Y5og+A9gNL2NBUn5i5cLgENMASeIxBWaQ5U7glF20oelG6diTx6Z vu9w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=9ps7TuAQURpuLzr3Fia73pRGmFO7lqbm8nTlS3QkaOw=; b=MxxiwNoqDxGXv/xc0SLLrrDpCTTwwufjmcQ7qgB7L7EIgTfHvfPyF7xHtOE+TNo4MY lASRa//1bFtHZeJEd0mNzAKJBKFlOwXVjy2KsXf3R8Nglje52K2t8L6QQp678J6EuWo+ Kebev3vZs7s9H5ITNzyDuHRMYrcits7T4iD32GFbbkLcxAPXhOIDlUZlBAOJnP5tIIGw GHBjMje8Z8Cn/NsJxaKGS1vYnn1SWFSy8qcoNQ3uXtv/YR9gvsEPJSGJ9HCMJKrGC6yC Bkuy6gf+8XyP1j3sor0zsj4hwTLYSFed1U8ep1t0M/1VuapQE3LeXyS4Bo0K1Tr0+u8W 6nMA== X-Gm-Message-State: AOAM531EcaIX4A+S6Kx9BF6sl1e5p5y/PVFmrLlmDv7w0TZC/BHBeu0D s26MUSXJsfoUjbqMMtGy+I81LXWASxc= X-Received: by 2002:a17:902:cb13:: with SMTP id c19mr1489714ply.51.1591146828895; Tue, 02 Jun 2020 18:13:48 -0700 (PDT) Received: from localhost.localdomain (174-21-143-238.tukw.qwest.net. [174.21.143.238]) by smtp.gmail.com with ESMTPSA id 3sm290067pfe.85.2020.06.02.18.13.47 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 02 Jun 2020 18:13:48 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v7 23/42] target/arm: Add gen_mte_check1 Date: Tue, 2 Jun 2020 18:12:58 -0700 Message-Id: <20200603011317.473934-24-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200603011317.473934-1-richard.henderson@linaro.org> References: <20200603011317.473934-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::643; envelope-from=richard.henderson@linaro.org; helo=mail-pl1-x643.google.com X-detected-operating-system: by eggs.gnu.org: No matching host in p0f cache. That's all we know. X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: peter.maydell@linaro.org, qemu-arm@nongnu.org, steplong@quicinc.com Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" Replace existing uses of check_data_tbi in translate-a64.c that perform a single logical memory access. Leave the helper blank for now to reduce the patch size. Signed-off-by: Richard Henderson --- target/arm/helper-a64.h | 1 + target/arm/internals.h | 8 ++++ target/arm/translate-a64.h | 2 + target/arm/mte_helper.c | 8 ++++ target/arm/translate-a64.c | 96 ++++++++++++++++++++++++++++---------- 5 files changed, 91 insertions(+), 24 deletions(-) -- 2.25.1 Reviewed-by: Peter Maydell diff --git a/target/arm/helper-a64.h b/target/arm/helper-a64.h index 7b628d100e..2faa49d0a3 100644 --- a/target/arm/helper-a64.h +++ b/target/arm/helper-a64.h @@ -104,6 +104,7 @@ DEF_HELPER_FLAGS_3(autdb, TCG_CALL_NO_WG, i64, env, i64, i64) DEF_HELPER_FLAGS_2(xpaci, TCG_CALL_NO_RWG_SE, i64, env, i64) DEF_HELPER_FLAGS_2(xpacd, TCG_CALL_NO_RWG_SE, i64, env, i64) +DEF_HELPER_FLAGS_3(mte_check1, TCG_CALL_NO_WG, i64, env, i32, i64) DEF_HELPER_FLAGS_3(irg, TCG_CALL_NO_RWG, i64, env, i64, i64) DEF_HELPER_FLAGS_4(addsubg, TCG_CALL_NO_RWG_SE, i64, env, i64, s32, i32) DEF_HELPER_FLAGS_3(ldg, TCG_CALL_NO_WG, i64, env, i64, i64) diff --git a/target/arm/internals.h b/target/arm/internals.h index 7c9abbabc9..fb92ef6b84 100644 --- a/target/arm/internals.h +++ b/target/arm/internals.h @@ -1310,6 +1310,14 @@ void arm_log_exception(int idx); #define LOG2_TAG_GRANULE 4 #define TAG_GRANULE (1 << LOG2_TAG_GRANULE) +/* Bits within a descriptor passed to the helper_mte_check* functions. */ +FIELD(MTEDESC, MIDX, 0, 4) +FIELD(MTEDESC, TBI, 4, 2) +FIELD(MTEDESC, TCMA, 6, 2) +FIELD(MTEDESC, WRITE, 8, 1) +FIELD(MTEDESC, ESIZE, 9, 5) +FIELD(MTEDESC, TSIZE, 14, 10) /* mte_checkN only */ + static inline int allocation_tag_from_addr(uint64_t ptr) { return extract64(ptr, 56, 4); diff --git a/target/arm/translate-a64.h b/target/arm/translate-a64.h index f02fbb63a4..1ab033e4c1 100644 --- a/target/arm/translate-a64.h +++ b/target/arm/translate-a64.h @@ -40,6 +40,8 @@ TCGv_ptr get_fpstatus_ptr(bool); bool logic_imm_decode_wmask(uint64_t *result, unsigned int immn, unsigned int imms, unsigned int immr); bool sve_access_check(DisasContext *s); +TCGv_i64 gen_mte_check1(DisasContext *s, TCGv_i64 addr, bool is_write, + bool tag_checked, int log2_size); /* We should have at some point before trying to access an FP register * done the necessary access check, so assert that diff --git a/target/arm/mte_helper.c b/target/arm/mte_helper.c index 27d4b4536c..ec12768dfc 100644 --- a/target/arm/mte_helper.c +++ b/target/arm/mte_helper.c @@ -358,3 +358,11 @@ void HELPER(stzgm_tags)(CPUARMState *env, uint64_t ptr, uint64_t val) memset(mem, tag_pair, tag_bytes); } } + +/* + * Perform an MTE checked access for a single logical or atomic access. + */ +uint64_t HELPER(mte_check1)(CPUARMState *env, uint32_t desc, uint64_t ptr) +{ + return ptr; +} diff --git a/target/arm/translate-a64.c b/target/arm/translate-a64.c index c4f86c69ba..9f8ae99cc0 100644 --- a/target/arm/translate-a64.c +++ b/target/arm/translate-a64.c @@ -204,20 +204,20 @@ static void gen_a64_set_pc(DisasContext *s, TCGv_i64 src) } /* - * Return a "clean" address for ADDR according to TBID. - * This is always a fresh temporary, as we need to be able to - * increment this independently of a dirty write-back address. + * Handle MTE and/or TBI. + * + * For TBI, ideally, we would do nothing. Proper behaviour on fault is + * for the tag to be present in the FAR_ELx register. But for user-only + * mode, we do not have a TLB with which to implement this, so we must + * remote the top byte now. + * + * Always return a fresh temporary that we can increment independently + * of the write-back address. */ + static TCGv_i64 clean_data_tbi(DisasContext *s, TCGv_i64 addr) { TCGv_i64 clean = new_tmp_a64(s); - /* - * In order to get the correct value in the FAR_ELx register, - * we must present the memory subsystem with the "dirty" address - * including the TBI. In system mode we can make this work via - * the TLB, dropping the TBI during translation. But for user-only - * mode we don't have that option, and must remove the top byte now. - */ #ifdef CONFIG_USER_ONLY gen_top_byte_ignore(s, clean, addr, s->tbid); #else @@ -245,6 +245,45 @@ static void gen_probe_access(DisasContext *s, TCGv_i64 ptr, tcg_temp_free_i32(t_size); } +/* + * For MTE, check a single logical or atomic access. This probes a single + * address, the exact one specified. The size and alignment of the access + * is not relevant to MTE, per se, but watchpoints do require the size, + * and we want to recognize those before making any other changes to state. + */ +static TCGv_i64 gen_mte_check1_mmuidx(DisasContext *s, TCGv_i64 addr, + bool is_write, bool tag_checked, + int log2_size, bool is_unpriv, + int core_idx) +{ + if (tag_checked && s->mte_active[is_unpriv]) { + TCGv_i32 tcg_desc; + TCGv_i64 ret; + int desc = 0; + + desc = FIELD_DP32(desc, MTEDESC, MIDX, core_idx); + desc = FIELD_DP32(desc, MTEDESC, TBI, s->tbid); + desc = FIELD_DP32(desc, MTEDESC, TCMA, s->tcma); + desc = FIELD_DP32(desc, MTEDESC, WRITE, is_write); + desc = FIELD_DP32(desc, MTEDESC, ESIZE, 1 << log2_size); + tcg_desc = tcg_const_i32(desc); + + ret = new_tmp_a64(s); + gen_helper_mte_check1(ret, cpu_env, tcg_desc, addr); + tcg_temp_free_i32(tcg_desc); + + return ret; + } + return clean_data_tbi(s, addr); +} + +TCGv_i64 gen_mte_check1(DisasContext *s, TCGv_i64 addr, bool is_write, + bool tag_checked, int log2_size) +{ + return gen_mte_check1_mmuidx(s, addr, is_write, tag_checked, log2_size, + false, get_mem_index(s)); +} + typedef struct DisasCompare64 { TCGCond cond; TCGv_i64 value; @@ -2358,7 +2397,7 @@ static void gen_compare_and_swap(DisasContext *s, int rs, int rt, if (rn == 31) { gen_check_sp_alignment(s); } - clean_addr = clean_data_tbi(s, cpu_reg_sp(s, rn)); + clean_addr = gen_mte_check1(s, cpu_reg_sp(s, rn), true, rn != 31, size); tcg_gen_atomic_cmpxchg_i64(tcg_rs, clean_addr, tcg_rs, tcg_rt, memidx, size | MO_ALIGN | s->be_data); } @@ -2376,7 +2415,9 @@ static void gen_compare_and_swap_pair(DisasContext *s, int rs, int rt, if (rn == 31) { gen_check_sp_alignment(s); } - clean_addr = clean_data_tbi(s, cpu_reg_sp(s, rn)); + + /* This is a single atomic access, despite the "pair". */ + clean_addr = gen_mte_check1(s, cpu_reg_sp(s, rn), true, rn != 31, size + 1); if (size == 2) { TCGv_i64 cmp = tcg_temp_new_i64(); @@ -2501,7 +2542,7 @@ static void disas_ldst_excl(DisasContext *s, uint32_t insn) if (is_lasr) { tcg_gen_mb(TCG_MO_ALL | TCG_BAR_STRL); } - clean_addr = clean_data_tbi(s, cpu_reg_sp(s, rn)); + clean_addr = gen_mte_check1(s, cpu_reg_sp(s, rn), true, rn != 31, size); gen_store_exclusive(s, rs, rt, rt2, clean_addr, size, false); return; @@ -2510,7 +2551,7 @@ static void disas_ldst_excl(DisasContext *s, uint32_t insn) if (rn == 31) { gen_check_sp_alignment(s); } - clean_addr = clean_data_tbi(s, cpu_reg_sp(s, rn)); + clean_addr = gen_mte_check1(s, cpu_reg_sp(s, rn), false, rn != 31, size); s->is_ldex = true; gen_load_exclusive(s, rt, rt2, clean_addr, size, false); if (is_lasr) { @@ -2530,7 +2571,7 @@ static void disas_ldst_excl(DisasContext *s, uint32_t insn) gen_check_sp_alignment(s); } tcg_gen_mb(TCG_MO_ALL | TCG_BAR_STRL); - clean_addr = clean_data_tbi(s, cpu_reg_sp(s, rn)); + clean_addr = gen_mte_check1(s, cpu_reg_sp(s, rn), true, rn != 31, size); do_gpr_st(s, cpu_reg(s, rt), clean_addr, size, true, rt, disas_ldst_compute_iss_sf(size, false, 0), is_lasr); return; @@ -2546,7 +2587,7 @@ static void disas_ldst_excl(DisasContext *s, uint32_t insn) if (rn == 31) { gen_check_sp_alignment(s); } - clean_addr = clean_data_tbi(s, cpu_reg_sp(s, rn)); + clean_addr = gen_mte_check1(s, cpu_reg_sp(s, rn), false, rn != 31, size); do_gpr_ld(s, cpu_reg(s, rt), clean_addr, size, false, false, true, rt, disas_ldst_compute_iss_sf(size, false, 0), is_lasr); tcg_gen_mb(TCG_MO_ALL | TCG_BAR_LDAQ); @@ -2560,7 +2601,8 @@ static void disas_ldst_excl(DisasContext *s, uint32_t insn) if (is_lasr) { tcg_gen_mb(TCG_MO_ALL | TCG_BAR_STRL); } - clean_addr = clean_data_tbi(s, cpu_reg_sp(s, rn)); + clean_addr = gen_mte_check1(s, cpu_reg_sp(s, rn), + true, rn != 31, size); gen_store_exclusive(s, rs, rt, rt2, clean_addr, size, true); return; } @@ -2578,7 +2620,8 @@ static void disas_ldst_excl(DisasContext *s, uint32_t insn) if (rn == 31) { gen_check_sp_alignment(s); } - clean_addr = clean_data_tbi(s, cpu_reg_sp(s, rn)); + clean_addr = gen_mte_check1(s, cpu_reg_sp(s, rn), + false, rn != 31, size); s->is_ldex = true; gen_load_exclusive(s, rt, rt2, clean_addr, size, true); if (is_lasr) { @@ -2872,6 +2915,7 @@ static void disas_ldst_reg_imm9(DisasContext *s, uint32_t insn, bool iss_valid = !is_vector; bool post_index; bool writeback; + int memidx; TCGv_i64 clean_addr, dirty_addr; @@ -2929,7 +2973,11 @@ static void disas_ldst_reg_imm9(DisasContext *s, uint32_t insn, if (!post_index) { tcg_gen_addi_i64(dirty_addr, dirty_addr, imm9); } - clean_addr = clean_data_tbi(s, dirty_addr); + + memidx = is_unpriv ? get_a64_user_mem_index(s) : get_mem_index(s); + clean_addr = gen_mte_check1_mmuidx(s, dirty_addr, is_store, + writeback || rn != 31, + size, is_unpriv, memidx); if (is_vector) { if (is_store) { @@ -2939,7 +2987,6 @@ static void disas_ldst_reg_imm9(DisasContext *s, uint32_t insn, } } else { TCGv_i64 tcg_rt = cpu_reg(s, rt); - int memidx = is_unpriv ? get_a64_user_mem_index(s) : get_mem_index(s); bool iss_sf = disas_ldst_compute_iss_sf(size, is_signed, opc); if (is_store) { @@ -3036,7 +3083,7 @@ static void disas_ldst_reg_roffset(DisasContext *s, uint32_t insn, ext_and_shift_reg(tcg_rm, tcg_rm, opt, shift ? size : 0); tcg_gen_add_i64(dirty_addr, dirty_addr, tcg_rm); - clean_addr = clean_data_tbi(s, dirty_addr); + clean_addr = gen_mte_check1(s, dirty_addr, is_store, true, size); if (is_vector) { if (is_store) { @@ -3121,7 +3168,7 @@ static void disas_ldst_reg_unsigned_imm(DisasContext *s, uint32_t insn, dirty_addr = read_cpu_reg_sp(s, rn, 1); offset = imm12 << size; tcg_gen_addi_i64(dirty_addr, dirty_addr, offset); - clean_addr = clean_data_tbi(s, dirty_addr); + clean_addr = gen_mte_check1(s, dirty_addr, is_store, rn != 31, size); if (is_vector) { if (is_store) { @@ -3214,7 +3261,7 @@ static void disas_ldst_atomic(DisasContext *s, uint32_t insn, if (rn == 31) { gen_check_sp_alignment(s); } - clean_addr = clean_data_tbi(s, cpu_reg_sp(s, rn)); + clean_addr = gen_mte_check1(s, cpu_reg_sp(s, rn), false, rn != 31, size); if (o3_opc == 014) { /* @@ -3291,7 +3338,8 @@ static void disas_ldst_pac(DisasContext *s, uint32_t insn, tcg_gen_addi_i64(dirty_addr, dirty_addr, offset); /* Note that "clean" and "dirty" here refer to TBI not PAC. */ - clean_addr = clean_data_tbi(s, dirty_addr); + clean_addr = gen_mte_check1(s, dirty_addr, false, + is_wback || rn != 31, size); tcg_rt = cpu_reg(s, rt); do_gpr_ld(s, tcg_rt, clean_addr, size, /* is_signed */ false, From patchwork Wed Jun 3 01:12:59 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 187427 Delivered-To: patch@linaro.org Received: by 2002:a92:cf06:0:0:0:0:0 with SMTP id c6csp77127ilo; Tue, 2 Jun 2020 18:26:42 -0700 (PDT) X-Google-Smtp-Source: ABdhPJxprPWAqiyOMNEfsPblktKkuRFLo1nEigCbspa5VDg5NSEsEoOfOi3VNQ8xPgDX7f9kLFbc X-Received: by 2002:a25:9944:: with SMTP id n4mr46339183ybo.404.1591147602176; Tue, 02 Jun 2020 18:26:42 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1591147602; cv=none; d=google.com; s=arc-20160816; b=lHuNFgqhua7gEdUdL9w4YiA+YsqvHjcf21ucVU9hkxE7SU9WZ2bBkrBVnfky72XAwY H+8THgR3NQEvKYZUvN7kpuyOD4KYi4AV049kPAdETsNoPEHJi0umJQ1tQzlZXDC6+aeB Wf8rzjW1Um3JmKUDpNfz5CMYPPvi2BatnViFI0PThgwMtYso/uqxA4seLLytecEfDNxM 3xL2dXWExy1HgZ7LDdYXYHIsV9v2n8nUXBKjpeDITWY2ZbWbNfZ4Gxb+eRuSSTFJvAX+ 8TvF8b/AOj87hazi09lwQ4gQvYT6MOJ69MoGoSv9w7Y02ZQJuyouzjL6745D5WsCSxmt tlhw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=/Mw9HZ5O96PBG/BuNXk71tQFvBrPmiee6xG1sg1z+/Y=; b=opiyjNdgpyfif1oBUIzIFDzO0TKYl1QMrYDmb/Ft1lg9Tt8fHovrYjVodcpnBNjK0a gJp3Ag6xNJQYze0uQ2PL+R57Lp7BcTslVSDW5xkzUYKrs0YAGq3gdbsJ63/Y2fdWuOoz 6JCnQUYJEZInpchyAmZfKMMb9MB13aqlCYuplcgzl5ajGWKQLL04ywvhGcgYGZ7qOi9w MOvD1cj9qxGv19By/DX0TLZKtwf1Gqn+1gymg+AVZ5J81m40gsVtftqdfR38m6K5lh6k TZQ13Ykf6R1X6p5p5fiKlpQ2W3n9bvKbvB1R77QdBEDoPaU5aWMzWrhBJ4vhLxXFSq+t LrZg== ARC-Authentication-Results: i=1; mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=HtWNy+DI; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id w14si331985ybq.278.2020.06.02.18.26.42 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 02 Jun 2020 18:26:42 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=HtWNy+DI; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:47230 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jgIAz-0002EH-Jx for patch@linaro.org; Tue, 02 Jun 2020 21:26:41 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:48846) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jgHyc-0004La-Sf for qemu-devel@nongnu.org; Tue, 02 Jun 2020 21:13:54 -0400 Received: from mail-pg1-x52c.google.com ([2607:f8b0:4864:20::52c]:44227) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1jgHyZ-0003cW-UF for qemu-devel@nongnu.org; Tue, 02 Jun 2020 21:13:54 -0400 Received: by mail-pg1-x52c.google.com with SMTP id p30so525967pgl.11 for ; Tue, 02 Jun 2020 18:13:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=/Mw9HZ5O96PBG/BuNXk71tQFvBrPmiee6xG1sg1z+/Y=; b=HtWNy+DIUGY37xvd4nPRKltCFg9fCFaSHu13nSSlrjXK/M+l8/GyZjsvhPezBO2Fu7 uLVuE8CId9wtwXmh33qtthl+aWTX+ZDjgyCraMUphbZIKjfzTgwBUP5QOO+B11t8Ig/8 ueiMWcR5pauT7Q2S0QDOFZsbsWUmCnQ5bCn8cjDl3wC2A/Mg1SQbR1LzJCIfmRGc8Lig zyCSfwBZG3RHGpdSQ3/W6JBzHGfpyC2SR8YNISofYke7gQrQJHPE0GUH+MbhK3e1Hgb+ ihnl05lVo2TY35PZNBScLrwb1UnVIeYPtXKKxBxSYuQleSDsISQJI1APPGgF5bguQ9gD X7nQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=/Mw9HZ5O96PBG/BuNXk71tQFvBrPmiee6xG1sg1z+/Y=; b=X8nO/bDKJRpPPV/+NdQpnWfGlpqMtFndmHPCITj3QDw5SQW7hWQ4yKkIPmBy9gpC3J PPtgPuCi6zSxaoFd3UrHitG3WDAGcOBt+Nj0fVKIx+BLvgOLjg9rqKijHU7sLxbFZ3zw k7mqtR2bKBi5XX6Xs5sxnI9aIEaEsmUbVJiCxZDFXfZSSgTc84lM0ILYLHfLisdwS16F RJxWMyiEI8pNlnj2+wOIwAr29FXnZO7qhYKUT3b6s0/h06maxiIGp1YHepCMxmk8UoQJ FRbjtbimcbXuzo5eRmqDiVx95cc4Qf85OrfvFvkvBTK3B0KsKYOa0dY2+xw+Zm8AqDlw tEyg== X-Gm-Message-State: AOAM533epCCdLBxDrgtK/oWBe4+G7CJOjcUjyAamboNey+GGkceujQzN CWczC/RMTOljvDYbhyV5SvsgxR+LJtI= X-Received: by 2002:a17:90a:df82:: with SMTP id p2mr2315787pjv.217.1591146830059; Tue, 02 Jun 2020 18:13:50 -0700 (PDT) Received: from localhost.localdomain (174-21-143-238.tukw.qwest.net. [174.21.143.238]) by smtp.gmail.com with ESMTPSA id 3sm290067pfe.85.2020.06.02.18.13.49 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 02 Jun 2020 18:13:49 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v7 24/42] target/arm: Add gen_mte_checkN Date: Tue, 2 Jun 2020 18:12:59 -0700 Message-Id: <20200603011317.473934-25-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200603011317.473934-1-richard.henderson@linaro.org> References: <20200603011317.473934-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::52c; envelope-from=richard.henderson@linaro.org; helo=mail-pg1-x52c.google.com X-detected-operating-system: by eggs.gnu.org: No matching host in p0f cache. That's all we know. X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: peter.maydell@linaro.org, qemu-arm@nongnu.org, steplong@quicinc.com Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" Replace existing uses of check_data_tbi in translate-a64.c that perform multiple logical memory access. Leave the helper blank for now to reduce the patch size. Signed-off-by: Richard Henderson --- target/arm/helper-a64.h | 1 + target/arm/translate-a64.h | 2 ++ target/arm/mte_helper.c | 8 +++++ target/arm/translate-a64.c | 71 +++++++++++++++++++++++++++++--------- 4 files changed, 66 insertions(+), 16 deletions(-) -- 2.25.1 Reviewed-by: Peter Maydell diff --git a/target/arm/helper-a64.h b/target/arm/helper-a64.h index 2faa49d0a3..005af678c7 100644 --- a/target/arm/helper-a64.h +++ b/target/arm/helper-a64.h @@ -105,6 +105,7 @@ DEF_HELPER_FLAGS_2(xpaci, TCG_CALL_NO_RWG_SE, i64, env, i64) DEF_HELPER_FLAGS_2(xpacd, TCG_CALL_NO_RWG_SE, i64, env, i64) DEF_HELPER_FLAGS_3(mte_check1, TCG_CALL_NO_WG, i64, env, i32, i64) +DEF_HELPER_FLAGS_3(mte_checkN, TCG_CALL_NO_WG, i64, env, i32, i64) DEF_HELPER_FLAGS_3(irg, TCG_CALL_NO_RWG, i64, env, i64, i64) DEF_HELPER_FLAGS_4(addsubg, TCG_CALL_NO_RWG_SE, i64, env, i64, s32, i32) DEF_HELPER_FLAGS_3(ldg, TCG_CALL_NO_WG, i64, env, i64, i64) diff --git a/target/arm/translate-a64.h b/target/arm/translate-a64.h index 1ab033e4c1..d0f2e131b3 100644 --- a/target/arm/translate-a64.h +++ b/target/arm/translate-a64.h @@ -42,6 +42,8 @@ bool logic_imm_decode_wmask(uint64_t *result, unsigned int immn, bool sve_access_check(DisasContext *s); TCGv_i64 gen_mte_check1(DisasContext *s, TCGv_i64 addr, bool is_write, bool tag_checked, int log2_size); +TCGv_i64 gen_mte_checkN(DisasContext *s, TCGv_i64 addr, bool is_write, + bool tag_checked, int count, int log2_esize); /* We should have at some point before trying to access an FP register * done the necessary access check, so assert that diff --git a/target/arm/mte_helper.c b/target/arm/mte_helper.c index ec12768dfc..907a12b366 100644 --- a/target/arm/mte_helper.c +++ b/target/arm/mte_helper.c @@ -366,3 +366,11 @@ uint64_t HELPER(mte_check1)(CPUARMState *env, uint32_t desc, uint64_t ptr) { return ptr; } + +/* + * Perform an MTE checked access for multiple logical accesses. + */ +uint64_t HELPER(mte_checkN)(CPUARMState *env, uint32_t desc, uint64_t ptr) +{ + return ptr; +} diff --git a/target/arm/translate-a64.c b/target/arm/translate-a64.c index 9f8ae99cc0..f40eee5a14 100644 --- a/target/arm/translate-a64.c +++ b/target/arm/translate-a64.c @@ -284,6 +284,34 @@ TCGv_i64 gen_mte_check1(DisasContext *s, TCGv_i64 addr, bool is_write, false, get_mem_index(s)); } +/* + * For MTE, check multiple logical sequential accesses. + */ +TCGv_i64 gen_mte_checkN(DisasContext *s, TCGv_i64 addr, bool is_write, + bool tag_checked, int log2_esize, int total_size) +{ + if (tag_checked && s->mte_active[0] && total_size != (1 << log2_esize)) { + TCGv_i32 tcg_desc; + TCGv_i64 ret; + int desc = 0; + + desc = FIELD_DP32(desc, MTEDESC, MIDX, get_mem_index(s)); + desc = FIELD_DP32(desc, MTEDESC, TBI, s->tbid); + desc = FIELD_DP32(desc, MTEDESC, TCMA, s->tcma); + desc = FIELD_DP32(desc, MTEDESC, WRITE, is_write); + desc = FIELD_DP32(desc, MTEDESC, ESIZE, 1 << log2_esize); + desc = FIELD_DP32(desc, MTEDESC, TSIZE, total_size); + tcg_desc = tcg_const_i32(desc); + + ret = new_tmp_a64(s); + gen_helper_mte_checkN(ret, cpu_env, tcg_desc, addr); + tcg_temp_free_i32(tcg_desc); + + return ret; + } + return gen_mte_check1(s, addr, is_write, tag_checked, log2_esize); +} + typedef struct DisasCompare64 { TCGCond cond; TCGv_i64 value; @@ -2835,7 +2863,10 @@ static void disas_ldst_pair(DisasContext *s, uint32_t insn) } } - clean_addr = clean_data_tbi(s, dirty_addr); + clean_addr = gen_mte_checkN(s, dirty_addr, !is_load, + (wback || rn != 31) && !set_tag, + size, 2 << size); + if (is_vector) { if (is_load) { do_fp_ld(s, rt, clean_addr, size); @@ -3501,7 +3532,7 @@ static void disas_ldst_multiple_struct(DisasContext *s, uint32_t insn) TCGv_i64 clean_addr, tcg_rn, tcg_ebytes; MemOp endian = s->be_data; - int ebytes; /* bytes per element */ + int total; /* total bytes */ int elements; /* elements per vector */ int rpt; /* num iterations */ int selem; /* structure elements */ @@ -3571,19 +3602,26 @@ static void disas_ldst_multiple_struct(DisasContext *s, uint32_t insn) endian = MO_LE; } - /* Consecutive little-endian elements from a single register + total = rpt * selem * (is_q ? 16 : 8); + tcg_rn = cpu_reg_sp(s, rn); + + /* + * Issue the MTE check vs the logical repeat count, before we + * promote consecutive little-endian elements below. + */ + clean_addr = gen_mte_checkN(s, tcg_rn, is_store, is_postidx || rn != 31, + size, total); + + /* + * Consecutive little-endian elements from a single register * can be promoted to a larger little-endian operation. */ if (selem == 1 && endian == MO_LE) { size = 3; } - ebytes = 1 << size; - elements = (is_q ? 16 : 8) / ebytes; - - tcg_rn = cpu_reg_sp(s, rn); - clean_addr = clean_data_tbi(s, tcg_rn); - tcg_ebytes = tcg_const_i64(ebytes); + elements = (is_q ? 16 : 8) >> size; + tcg_ebytes = tcg_const_i64(1 << size); for (r = 0; r < rpt; r++) { int e; for (e = 0; e < elements; e++) { @@ -3617,7 +3655,7 @@ static void disas_ldst_multiple_struct(DisasContext *s, uint32_t insn) if (is_postidx) { if (rm == 31) { - tcg_gen_addi_i64(tcg_rn, tcg_rn, rpt * elements * selem * ebytes); + tcg_gen_addi_i64(tcg_rn, tcg_rn, total); } else { tcg_gen_add_i64(tcg_rn, tcg_rn, cpu_reg(s, rm)); } @@ -3663,7 +3701,7 @@ static void disas_ldst_single_struct(DisasContext *s, uint32_t insn) int selem = (extract32(opc, 0, 1) << 1 | R) + 1; bool replicate = false; int index = is_q << 3 | S << 2 | size; - int ebytes, xs; + int xs, total; TCGv_i64 clean_addr, tcg_rn, tcg_ebytes; if (extract32(insn, 31, 1)) { @@ -3717,16 +3755,17 @@ static void disas_ldst_single_struct(DisasContext *s, uint32_t insn) return; } - ebytes = 1 << scale; - if (rn == 31) { gen_check_sp_alignment(s); } + total = selem << scale; tcg_rn = cpu_reg_sp(s, rn); - clean_addr = clean_data_tbi(s, tcg_rn); - tcg_ebytes = tcg_const_i64(ebytes); + clean_addr = gen_mte_checkN(s, tcg_rn, !is_load, is_postidx || rn != 31, + scale, total); + + tcg_ebytes = tcg_const_i64(1 << scale); for (xs = 0; xs < selem; xs++) { if (replicate) { /* Load and replicate to all elements */ @@ -3753,7 +3792,7 @@ static void disas_ldst_single_struct(DisasContext *s, uint32_t insn) if (is_postidx) { if (rm == 31) { - tcg_gen_addi_i64(tcg_rn, tcg_rn, selem * ebytes); + tcg_gen_addi_i64(tcg_rn, tcg_rn, total); } else { tcg_gen_add_i64(tcg_rn, tcg_rn, cpu_reg(s, rm)); } From patchwork Wed Jun 3 01:13:00 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 187429 Delivered-To: patch@linaro.org Received: by 2002:a92:cf06:0:0:0:0:0 with SMTP id c6csp78001ilo; Tue, 2 Jun 2020 18:28:14 -0700 (PDT) X-Google-Smtp-Source: ABdhPJwYqDQkZyYMnTUDg7RdPMOENoNCHgtMFHqQVr8LKdPEe9FRUe1il4AfMPJUwGzUSmiLfu6C X-Received: by 2002:a25:6ad5:: with SMTP id f204mr42951659ybc.147.1591147694262; Tue, 02 Jun 2020 18:28:14 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1591147694; cv=none; d=google.com; s=arc-20160816; b=LKhjkC8v1FaxIZEcNVPKnKu45kUAYCQIUVpS5wNjliuvs+API0QfZs8NbrhxQrV5I0 dBrrlxu4etxxjvpd7Yirk/6EskY6fFITGmVWkr87i3kFdHM+5T4iiU65dgelHi0Vxfjh FUfbr/SQ+dGQcyFIxsio/QpWBlQBaT7yhoJE83T/3HgC7XrAyaRdwawoI03gwUv4M9dR fr+EhKk56lYckMh0rhwObOthUMR9ZrY8j31XxYBxkFN39yZAd3opHgn24Pwj/xFGkuvy vS+pgXywAwwRpDIynyzMVM+2++TxLUPiw8DuEeRYoEHC1Hw7cSi2SRrMTJrk6FE/LyHD pDMQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=5YykGMNydL05PlFQms9HDJyH0QWODtn6c8zGI4QHg9g=; b=j8FdAtxd6rhKwIqPHtFvmA6fWztvx2oEAtby/og8eRyDDfUZDEuiQY7IK77dGLVi+R qp5zyJNj/R6rBWaOp5+xZ40SDtVvG8kCvYLMAkxqp61zfQWIrXHtHptfeg5ONOMMf/3M l/ysqbSIqXGAtShy1rmXNFTj64Yx+DqWsPGu9xTkbxy6TUYYDyGG31YWZsxEHuljWSuD eDh6///Br6Lx85Wft3WmqJpVEDY5KO161PRSsJvSAm8EJ30anT7hUeDkrB2CjL1qOe88 VrYVs0g/0ELg0V59ytznzf8XnfBYiNtpS0SIvqFXWMiutiGF67qrp1tACGBNzehIhfez 8D1Q== ARC-Authentication-Results: i=1; mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=Sq7Sl4eq; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id e71si5673ybb.448.2020.06.02.18.28.14 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 02 Jun 2020 18:28:14 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=Sq7Sl4eq; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:57016 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jgICT-0006Df-NN for patch@linaro.org; Tue, 02 Jun 2020 21:28:13 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:48870) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jgHye-0004QF-94 for qemu-devel@nongnu.org; Tue, 02 Jun 2020 21:13:56 -0400 Received: from mail-pl1-x641.google.com ([2607:f8b0:4864:20::641]:45834) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1jgHyb-0003cu-9F for qemu-devel@nongnu.org; Tue, 02 Jun 2020 21:13:55 -0400 Received: by mail-pl1-x641.google.com with SMTP id y11so187296plt.12 for ; Tue, 02 Jun 2020 18:13:52 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=5YykGMNydL05PlFQms9HDJyH0QWODtn6c8zGI4QHg9g=; b=Sq7Sl4eqcEPoPxVp9kCS/LB8QK16usl1F+KAs33VPWMSRcsBRsKpJ95MangK3GZNOc x0c+uYUNJaBCQYHEetS19z/ziWMIK3EjUyzLyQtQXWSBXBhPLMHJhpibcUuOOfIwFqCr u6ngI6+YPgdtnWR5peqFH3Dkj0RaA2DM6SD+paTjUhXvCmI4MXERT5+WgDVYppfwKykU 49uS9QbSEeCqc+HUCo9/1WNWr15byhxWTvcMrEht9t16njuAG9ggXriUuYqYNkR0JzMI XPzoXzoJIMAjcGKSwRWO79FB70jG/Rj66Gq0jzxXtKvw+jHHF5D30YZygrQpVWl0hnqM poIg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=5YykGMNydL05PlFQms9HDJyH0QWODtn6c8zGI4QHg9g=; b=i+r7x70ybzLDX7fg2IM/k0IiPr7k8GnJWY4cQZAJkNnJyocmJxzoHrgknUtY/JNa5g kwzNES5x+yFm7fL0tXrZ0rJDFFt01uVoiCxsGjBhZUl75Ww6oAfps9Xg5kdeb/RQAAhY iR8/QLxSbfKzAv5jsSE0a1JZ73Ds8MAADRn3f1uwZH1PvHIgw5utupYC1mHdctY9pgYb 1EaGeIEdPMihPSN2aV1EJld/i0g5BYS410+hFFBFYEvvn2VGVhDaQQT89wmHL5v4XG3u 8p0cL5TCCqj+YxlaO8J/qwH3QV04LN6GZyn5TsOzIVjIWPCY//J5y3oW8ori7Y7pE6wE 9PJg== X-Gm-Message-State: AOAM532M5Ph1ANJ2B/abptu9HFaT6z5Ab7tBoc7gaF2O6EIC/jRnyyW0 gxX+pzDBXsWKFucw31jtE202e/XcV/k= X-Received: by 2002:a17:902:6b49:: with SMTP id g9mr26349769plt.66.1591146831490; Tue, 02 Jun 2020 18:13:51 -0700 (PDT) Received: from localhost.localdomain (174-21-143-238.tukw.qwest.net. [174.21.143.238]) by smtp.gmail.com with ESMTPSA id 3sm290067pfe.85.2020.06.02.18.13.50 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 02 Jun 2020 18:13:50 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v7 25/42] target/arm: Implement helper_mte_check1 Date: Tue, 2 Jun 2020 18:13:00 -0700 Message-Id: <20200603011317.473934-26-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200603011317.473934-1-richard.henderson@linaro.org> References: <20200603011317.473934-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::641; envelope-from=richard.henderson@linaro.org; helo=mail-pl1-x641.google.com X-detected-operating-system: by eggs.gnu.org: No matching host in p0f cache. That's all we know. X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: peter.maydell@linaro.org, qemu-arm@nongnu.org, steplong@quicinc.com Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" Fill out the stub that was added earlier. Signed-off-by: Richard Henderson --- target/arm/internals.h | 47 +++++++++++++++ target/arm/mte_helper.c | 126 +++++++++++++++++++++++++++++++++++++++- 2 files changed, 172 insertions(+), 1 deletion(-) -- 2.25.1 diff --git a/target/arm/internals.h b/target/arm/internals.h index fb92ef6b84..8ae80f3945 100644 --- a/target/arm/internals.h +++ b/target/arm/internals.h @@ -1318,6 +1318,9 @@ FIELD(MTEDESC, WRITE, 8, 1) FIELD(MTEDESC, ESIZE, 9, 5) FIELD(MTEDESC, TSIZE, 14, 10) /* mte_checkN only */ +bool mte_probe1(CPUARMState *env, uint32_t desc, uint64_t ptr, uintptr_t ra); +uint64_t mte_check1(CPUARMState *env, uint32_t desc, uint64_t ptr, uintptr_t ra); + static inline int allocation_tag_from_addr(uint64_t ptr) { return extract64(ptr, 56, 4); @@ -1328,4 +1331,48 @@ static inline uint64_t address_with_allocation_tag(uint64_t ptr, int rtag) return deposit64(ptr, 56, 4, rtag); } +/* Return true if tbi bits mean that the access is checked. */ +static inline bool tbi_check(uint32_t desc, int bit55) +{ + return (desc >> (R_MTEDESC_TBI_SHIFT + bit55)) & 1; +} + +/* Return true if tcma bits mean that the access is unchecked. */ +static inline bool tcma_check(uint32_t desc, int bit55, int ptr_tag) +{ + /* + * We had extracted bit55 and ptr_tag for other reasons, so fold + * (ptr<59:55> == 00000 || ptr<59:55> == 11111) into a single test. + */ + bool match = ((ptr_tag + bit55) & 0xf) == 0; + bool tcma = (desc >> (R_MTEDESC_TCMA_SHIFT + bit55)) & 1; + return tcma && match; +} + +/* + * For TBI, ideally, we would do nothing. Proper behaviour on fault is + * for the tag to be present in the FAR_ELx register. But for user-only + * mode, we do not have a TLB with which to implement this, so we must + * remote the top byte. + */ +static inline uint64_t useronly_clean_ptr(uint64_t ptr) +{ + /* TBI is known to be enabled. */ +#ifdef CONFIG_USER_ONLY + ptr = sextract64(ptr, 0, 56); +#endif + return ptr; +} + +static inline uint64_t useronly_maybe_clean_ptr(uint32_t desc, uint64_t ptr) +{ +#ifdef CONFIG_USER_ONLY + int64_t clean_ptr = sextract64(ptr, 0, 56); + if (tbi_check(desc, clean_ptr < 0)) { + ptr = clean_ptr; + } +#endif + return ptr; +} + #endif diff --git a/target/arm/mte_helper.c b/target/arm/mte_helper.c index 907a12b366..72ff5543cf 100644 --- a/target/arm/mte_helper.c +++ b/target/arm/mte_helper.c @@ -359,12 +359,136 @@ void HELPER(stzgm_tags)(CPUARMState *env, uint64_t ptr, uint64_t val) } } +/* Record a tag check failure. */ +static void mte_check_fail(CPUARMState *env, int mmu_idx, + uint64_t dirty_ptr, uintptr_t ra) +{ + ARMMMUIdx arm_mmu_idx = core_to_aa64_mmu_idx(mmu_idx); + int el, reg_el, tcf, select; + uint64_t sctlr; + + reg_el = regime_el(env, arm_mmu_idx); + sctlr = env->cp15.sctlr_el[reg_el]; + + switch (arm_mmu_idx) { + case ARMMMUIdx_E10_0: + case ARMMMUIdx_E20_0: + el = 0; + tcf = extract64(sctlr, 38, 2); + break; + default: + el = reg_el; + tcf = extract64(sctlr, 40, 2); + } + + switch (tcf) { + case 1: + /* + * Tag check fail causes a synchronous exception. + * + * In restore_state_to_opc, we set the exception syndrome + * for the load or store operation. Unwind first so we + * may overwrite that with the syndrome for the tag check. + */ + cpu_restore_state(env_cpu(env), ra, true); + env->exception.vaddress = dirty_ptr; + raise_exception(env, EXCP_DATA_ABORT, + syn_data_abort_no_iss(el != 0, 0, 0, 0, 0, 0, 0x11), + exception_target_el(env)); + /* noreturn, but fall through to the assert anyway */ + + case 0: + /* + * Tag check fail does not affect the PE. + * We eliminate this case by not setting MTE_ACTIVE + * in tb_flags, so that we never make this runtime call. + */ + g_assert_not_reached(); + + case 2: + /* Tag check fail causes asynchronous flag set. */ + mmu_idx = arm_mmu_idx_el(env, el); + if (regime_has_2_ranges(mmu_idx)) { + select = extract64(dirty_ptr, 55, 1); + } else { + select = 0; + } + env->cp15.tfsr_el[el] |= 1 << select; + break; + + default: + /* Case 3: Reserved. */ + qemu_log_mask(LOG_GUEST_ERROR, + "Tag check failure with SCTLR_EL%d.TCF%s " + "set to reserved value %d\n", + reg_el, el ? "" : "0", tcf); + break; + } +} + /* * Perform an MTE checked access for a single logical or atomic access. */ +static bool mte_probe1_int(CPUARMState *env, uint32_t desc, uint64_t ptr, + uintptr_t ra, int bit55) +{ + int mem_tag, mmu_idx, ptr_tag, size; + MMUAccessType type; + uint8_t *mem; + + ptr_tag = allocation_tag_from_addr(ptr); + + if (tcma_check(desc, bit55, ptr_tag)) { + return true; + } + + mmu_idx = FIELD_EX32(desc, MTEDESC, MIDX); + type = FIELD_EX32(desc, MTEDESC, WRITE) ? MMU_DATA_STORE : MMU_DATA_LOAD; + size = FIELD_EX32(desc, MTEDESC, ESIZE); + + mem = allocation_tag_mem(env, mmu_idx, ptr, type, size, + MMU_DATA_LOAD, 1, ra); + if (!mem) { + return true; + } + + mem_tag = load_tag1(ptr, mem); + return ptr_tag == mem_tag; +} + +/* No-fault version of mte_check1, to be used by SVE for MemSingleNF. */ +bool mte_probe1(CPUARMState *env, uint32_t desc, uint64_t ptr, uintptr_t ra) +{ + int bit55 = extract64(ptr, 55, 1); + + /* If TBI is disabled, the access is unchecked. */ + if (unlikely(!tbi_check(desc, bit55))) { + return true; + } + + return mte_probe1_int(env, desc, ptr, ra, bit55); +} + +uint64_t mte_check1(CPUARMState *env, uint32_t desc, uint64_t ptr, uintptr_t ra) +{ + int bit55 = extract64(ptr, 55, 1); + + /* If TBI is disabled, the access is unchecked, and ptr is not dirty. */ + if (unlikely(!tbi_check(desc, bit55))) { + return ptr; + } + + if (unlikely(!mte_probe1_int(env, desc, ptr, ra, bit55))) { + int mmu_idx = FIELD_EX32(desc, MTEDESC, MIDX); + mte_check_fail(env, mmu_idx, ptr, ra); + } + + return useronly_clean_ptr(ptr); +} + uint64_t HELPER(mte_check1)(CPUARMState *env, uint32_t desc, uint64_t ptr) { - return ptr; + return mte_check1(env, desc, ptr, GETPC()); } /* From patchwork Wed Jun 3 01:13:01 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 187420 Delivered-To: patch@linaro.org Received: by 2002:a92:cf06:0:0:0:0:0 with SMTP id c6csp74656ilo; Tue, 2 Jun 2020 18:22:10 -0700 (PDT) X-Google-Smtp-Source: ABdhPJxSzOjR2owssyFguVZ862p30UxKTNwigdxcLHpYqUMRfF9rcfiM1DYdH748JMW4k6wNLZRN X-Received: by 2002:a25:9ac4:: with SMTP id t4mr23004804ybo.356.1591147329865; Tue, 02 Jun 2020 18:22:09 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1591147329; cv=none; d=google.com; s=arc-20160816; b=e/UBHRxrQcZw9KDlfdgUfjeKTg3F+yvPZWMyt9oXuL8VLcUV6STkKacMhJkE2E+K47 hPjV0vuULVtVFIF8lEqvy2aTx8tEbzGFg8ThXoHAjGUbNqIzGTz+EHLazT30P+3GxyE2 QmHjoJsDGch+SC+G5FfU1ckr7u5uGZ3+LiOEq+3DA9sChdsTJr2aM9O+i68gPnFkUA6m ItKCNmM75Z9K9JP6lZw/MhsaE6UEIbBwfJwDCTXjHffRKtdsWf39kpwN4TGCb0izKYRz o4wF6RcsFG4CJkrz/WwTUG/nOBJZH7iQ27DKM9wGKIMQaOpo0UMWPE7/DIley3HLZvlt YbzA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=+uzIV7dO8JDXOF8PjdnP8vAM1Vj2wACSpWbkwbf2z7U=; b=AK7JrQGi+yldtlfCIV3+jAjey9fE/LCZTlyncaoCT+n8Hkl0xAQfpqQXK8C+xFAAUt jBGKdvbrX/AEifv9VzgLyH8ATykOEdLlEpkt8IG3J6iw4lCZYrVBzS7vUvT2vin1C/4H V3CbJaJlQqmTcz0Q5elcttnuuDOIn/aI3oq/FZA1xuHcPzlw+hz61UiMdpc8OGgFfY6m lVCopxZNQKgM8dOQtpr2sIU8p2wpG6Kv08vbx4ShVH8qNRUbMM/f7CksB3QAgyHra8gJ /EUemgbNb9ayzSJj8svhSVUEHLPN6LJsG2xyZ0D6K9eTH4uCJKnbBvSfV28ufiKWYv8z txPQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=sSjsr+lT; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id x128si312835ybg.472.2020.06.02.18.22.09 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 02 Jun 2020 18:22:09 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=sSjsr+lT; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:57402 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jgI6b-0003Fp-CG for patch@linaro.org; Tue, 02 Jun 2020 21:22:09 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:48890) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jgHyf-0004Tq-CO for qemu-devel@nongnu.org; Tue, 02 Jun 2020 21:13:57 -0400 Received: from mail-pf1-x442.google.com ([2607:f8b0:4864:20::442]:39332) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1jgHyc-0003dG-OR for qemu-devel@nongnu.org; Tue, 02 Jun 2020 21:13:57 -0400 Received: by mail-pf1-x442.google.com with SMTP id d66so417483pfd.6 for ; Tue, 02 Jun 2020 18:13:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=+uzIV7dO8JDXOF8PjdnP8vAM1Vj2wACSpWbkwbf2z7U=; b=sSjsr+lTcFtP2ELmksFBwo00Fxek3vqLxh1hRKW31fRIIBna1iLOjNn9CWnIRBVu25 c7LtCRvwypAzYdkL+bzzd/3znmTgxy9Jbiv3r6V0IJ7P71pEKWGWHKAy0L4ax1xY40YX Y7OjTX/qJnBAMcUx6U6v3pcGNkdqcUSn/kXnH+RfMgfqKIjcf54pe9EiR9dV/DDaaDfW F5O/JTWEGYMbqpGzBuhH/Fv+fQ9dtG6JRB3P67p1/TEGqQ2YTRN1qw6bIDCriP6kICY7 nhp4Ldq4qEFhShN6fi3ziY9mshdehsodsgPUGQ939x0+xrRvqHdMQX7VzfVSk5aZBVS5 k0Gg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=+uzIV7dO8JDXOF8PjdnP8vAM1Vj2wACSpWbkwbf2z7U=; b=uIRFeXYFGsvBO1DC5iANcVflbQ7rMOLF6Z+OE6VOHzkL6pJ2lyvTVEViYW9o3w+itT 5vJ+IWu9VFQEAHGFvfWEpRasUN/Eij4l/JZ3ydJFkobI9P4zjlYzS2Nq/PLzsfEGF1xN KYdIechOE2qsu4nW0FwajUwWL9PQeAAokhTrIcB5CPnys0eSKl5WUnHf1P4DojPvqUB8 eauDdTdY6FlDYir7XMJmzwm5sp6czuCYp2MxQmhSuTHiXB+W3EzgOaDES41wrGzuy6gq +DXoqAIB07gAIeJYFeexv3DhDVEFNtZ7Uvg+EOnKiq4vIZ0vSEspTOVMmJsYP0dyl7LR 7hnw== X-Gm-Message-State: AOAM530SXatL49kvSrvU89Z4OzS6dC4+T8Sa85MvXOhVtA8rtYk0sCA+ 61dgmmDm8+pi9L6v9jL+ZeuTeYwaFMc= X-Received: by 2002:a65:5206:: with SMTP id o6mr25443433pgp.16.1591146832722; Tue, 02 Jun 2020 18:13:52 -0700 (PDT) Received: from localhost.localdomain (174-21-143-238.tukw.qwest.net. [174.21.143.238]) by smtp.gmail.com with ESMTPSA id 3sm290067pfe.85.2020.06.02.18.13.51 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 02 Jun 2020 18:13:52 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v7 26/42] target/arm: Implement helper_mte_checkN Date: Tue, 2 Jun 2020 18:13:01 -0700 Message-Id: <20200603011317.473934-27-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200603011317.473934-1-richard.henderson@linaro.org> References: <20200603011317.473934-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::442; envelope-from=richard.henderson@linaro.org; helo=mail-pf1-x442.google.com X-detected-operating-system: by eggs.gnu.org: No matching host in p0f cache. That's all we know. X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: peter.maydell@linaro.org, qemu-arm@nongnu.org, steplong@quicinc.com Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" Fill out the stub that was added earlier. Signed-off-by: Richard Henderson --- v7: Fix page crossing test (szabolcs nagy). --- target/arm/internals.h | 1 + target/arm/mte_helper.c | 165 +++++++++++++++++++++++++++++++++++++++- 2 files changed, 165 insertions(+), 1 deletion(-) -- 2.25.1 Reviewed-by: Peter Maydell diff --git a/target/arm/internals.h b/target/arm/internals.h index 8ae80f3945..cc485603a8 100644 --- a/target/arm/internals.h +++ b/target/arm/internals.h @@ -1320,6 +1320,7 @@ FIELD(MTEDESC, TSIZE, 14, 10) /* mte_checkN only */ bool mte_probe1(CPUARMState *env, uint32_t desc, uint64_t ptr, uintptr_t ra); uint64_t mte_check1(CPUARMState *env, uint32_t desc, uint64_t ptr, uintptr_t ra); +uint64_t mte_checkN(CPUARMState *env, uint32_t desc, uint64_t ptr, uintptr_t ra); static inline int allocation_tag_from_addr(uint64_t ptr) { diff --git a/target/arm/mte_helper.c b/target/arm/mte_helper.c index 72ff5543cf..c8a523ed4e 100644 --- a/target/arm/mte_helper.c +++ b/target/arm/mte_helper.c @@ -494,7 +494,170 @@ uint64_t HELPER(mte_check1)(CPUARMState *env, uint32_t desc, uint64_t ptr) /* * Perform an MTE checked access for multiple logical accesses. */ + +/** + * checkN: + * @tag: tag memory to test + * @odd: true to begin testing at tags at odd nibble + * @cmp: the tag to compare against + * @count: number of tags to test + * + * Return the number of successful tests. + * Thus a return value < @count indicates a failure. + * + * A note about sizes: count is expected to be small. + * + * The most common use will be LDP/STP of two integer registers, + * which means 16 bytes of memory touching at most 2 tags, but + * often the access is aligned and thus just 1 tag. + * + * Using AdvSIMD LD/ST (multiple), one can access 64 bytes of memory, + * touching at most 5 tags. SVE LDR/STR (vector) with the default + * vector length is also 64 bytes; the maximum architectural length + * is 256 bytes touching at most 9 tags. + * + * The loop below uses 7 logical operations and 1 memory operation + * per tag pair. An implementation that loads an aligned word and + * uses masking to ignore adjacent tags requires 18 logical operations + * and thus does not begin to pay off until 6 tags. + * Which, according to the survey above, is unlikely to be common. + */ + +static int checkN(uint8_t *mem, int odd, int cmp, int count) +{ + int n = 0, diff; + + /* Replicate the test tag and compare. */ + cmp *= 0x11; + diff = *mem++ ^ cmp; + + if (odd) { + goto start_odd; + } + + while (1) { + /* Test even tag. */ + if (unlikely((diff) & 0x0f)) { + break; + } + if (++n == count) { + break; + } + + start_odd: + /* Test odd tag. */ + if (unlikely((diff) & 0xf0)) { + break; + } + if (++n == count) { + break; + } + + diff = *mem++ ^ cmp; + } + return n; +} + +uint64_t mte_checkN(CPUARMState *env, uint32_t desc, uint64_t ptr, uintptr_t ra) +{ + int mmu_idx, ptr_tag, bit55; + uint64_t ptr_last, ptr_end, prev_page, next_page; + uint64_t tag_first, tag_end; + uint64_t tag_byte_first, tag_byte_end; + uint32_t esize, total, tag_count, tag_size, n, c; + uint8_t *mem1, *mem2; + MMUAccessType type; + + bit55 = extract64(ptr, 55, 1); + + /* If TBI is disabled, the access is unchecked, and ptr is not dirty. */ + if (unlikely(!tbi_check(desc, bit55))) { + return ptr; + } + + ptr_tag = allocation_tag_from_addr(ptr); + + if (tcma_check(desc, bit55, ptr_tag)) { + goto done; + } + + mmu_idx = FIELD_EX32(desc, MTEDESC, MIDX); + type = FIELD_EX32(desc, MTEDESC, WRITE) ? MMU_DATA_STORE : MMU_DATA_LOAD; + esize = FIELD_EX32(desc, MTEDESC, ESIZE); + total = FIELD_EX32(desc, MTEDESC, TSIZE); + + /* Find the addr of the end of the access, and of the last element. */ + ptr_end = ptr + total; + ptr_last = ptr_end - esize; + + /* Round the bounds to the tag granule, and compute the number of tags. */ + tag_first = QEMU_ALIGN_DOWN(ptr, TAG_GRANULE); + tag_end = QEMU_ALIGN_UP(ptr_last, TAG_GRANULE); + tag_count = (tag_end - tag_first) / TAG_GRANULE; + + /* Round the bounds to twice the tag granule, and compute the bytes. */ + tag_byte_first = QEMU_ALIGN_DOWN(ptr, 2 * TAG_GRANULE); + tag_byte_end = QEMU_ALIGN_UP(ptr_last, 2 * TAG_GRANULE); + + /* Locate the page boundaries. */ + prev_page = ptr & TARGET_PAGE_MASK; + next_page = prev_page + TARGET_PAGE_SIZE; + + if (likely(tag_end - prev_page <= TARGET_PAGE_SIZE)) { + /* Memory access stays on one page. */ + tag_size = (tag_byte_end - tag_byte_first) / (2 * TAG_GRANULE); + mem1 = allocation_tag_mem(env, mmu_idx, ptr, type, total, + MMU_DATA_LOAD, tag_size, ra); + if (!mem1) { + goto done; + } + /* Perform all of the comparisons. */ + n = checkN(mem1, ptr & TAG_GRANULE, ptr_tag, tag_count); + } else { + /* Memory access crosses to next page. */ + tag_size = (next_page - tag_byte_first) / (2 * TAG_GRANULE); + mem1 = allocation_tag_mem(env, mmu_idx, ptr, type, next_page - ptr, + MMU_DATA_LOAD, tag_size, ra); + + tag_size = (tag_byte_end - next_page) / (2 * TAG_GRANULE); + mem2 = allocation_tag_mem(env, mmu_idx, next_page, type, + ptr_end - next_page, + MMU_DATA_LOAD, tag_size, ra); + + /* + * Perform all of the comparisons. + * Note the possible but unlikely case of the operation spanning + * two pages that do not both have tagging enabled. + */ + n = c = (next_page - tag_first) / TAG_GRANULE; + if (mem1) { + n = checkN(mem1, ptr & TAG_GRANULE, ptr_tag, c); + } + if (n == c) { + if (!mem2) { + goto done; + } + n += checkN(mem2, 0, ptr_tag, tag_count - c); + } + } + + /* + * If we failed, we know which granule. Compute the element that + * is first in that granule, and signal failure on that element. + */ + if (unlikely(n < tag_count)) { + uint64_t fail_ofs; + + fail_ofs = tag_first + n * TAG_GRANULE - ptr; + fail_ofs = ROUND_UP(fail_ofs, esize); + mte_check_fail(env, mmu_idx, ptr + fail_ofs, ra); + } + + done: + return useronly_clean_ptr(ptr); +} + uint64_t HELPER(mte_checkN)(CPUARMState *env, uint32_t desc, uint64_t ptr) { - return ptr; + return mte_checkN(env, desc, ptr, GETPC()); } From patchwork Wed Jun 3 01:13:02 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 187424 Delivered-To: patch@linaro.org Received: by 2002:a92:cf06:0:0:0:0:0 with SMTP id c6csp75963ilo; Tue, 2 Jun 2020 18:24:36 -0700 (PDT) X-Google-Smtp-Source: ABdhPJxMkH2ClyYWRxMzPpJGpKGi43fNd91jqY9mCrTrGbPiTTb3g2D1iznTjzRARN+jjkXvND4t X-Received: by 2002:a25:d8d5:: with SMTP id p204mr33209632ybg.28.1591147476903; Tue, 02 Jun 2020 18:24:36 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1591147476; cv=none; d=google.com; s=arc-20160816; b=SeOw0mU8eSoCTh9whsE9a01+2YFOv+Ap1cldAweIS72ZVf2UKm2u+RGtI+H10LI4Vz nNopvvaHC1HEDUYpdRdqAZN11IXANzkj/fBpQWM79fIbZwNz0jRkk1kPC/T4rH1Zcopp pJu81Ko/cbbbzolyPkJqM3jIlRPU6Wtrol3971ZGPOk/G9iUobX2RGv9uB1UFzhK6ND2 Wv4d+y4eAE4NmLhpMkZ+4JugkSv2SsH+kWdSi2j5fhztvPAUG0nNLvRJ08/A/VUCwtDe XPVg6GuQwVu8bHlyeaUGu5PRdo+BtBuqkeIsRfNXRJkD0yULtye7xA84NLkI9Ynp6x62 pfLg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=hOZXrSo9cXF1wY4dgGHftC4urn5UGRE1IwLPUAUKWmU=; b=Ma3YbSWedtaMQ8M+ncjydQ0eEDDZIAtWWg7QV8pnPQH3sYVRogPKBTl8t3wHtaCiNX 4+/INxsrDW00aEj2jtdW0MKmmr68uy+QRW2LtR6Px1jkyIsrsbMfcn8OnwSWZw8SQQQP X6Kb9Zi5+QFbh6TtiTziBkt2RBGiPdOziZXYRGxJ9pdFekiOnZjoxoJM7k/lxcQietXr gdwzviAaRS69yeQ5gkAfmnid0FjQT0MosjRsfHDOXkhcYtD3P96av3N790UrXD2WZcH9 dEEE1fkWYRPIix9qw0h0yEH5YWNUCN+r0vchrmpgxYpsjMV6jGzrJgq5T6jgJRGI0jRq +h6Q== ARC-Authentication-Results: i=1; mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=MvVqF6OB; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id m11si349187ybm.322.2020.06.02.18.24.36 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 02 Jun 2020 18:24:36 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=MvVqF6OB; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:41854 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jgI8y-0008Qb-Be for patch@linaro.org; Tue, 02 Jun 2020 21:24:36 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:48884) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jgHyf-0004T1-4Q for qemu-devel@nongnu.org; Tue, 02 Jun 2020 21:13:57 -0400 Received: from mail-pg1-x542.google.com ([2607:f8b0:4864:20::542]:38987) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1jgHye-0003ds-7M for qemu-devel@nongnu.org; Tue, 02 Jun 2020 21:13:56 -0400 Received: by mail-pg1-x542.google.com with SMTP id w20so546267pga.6 for ; Tue, 02 Jun 2020 18:13:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=hOZXrSo9cXF1wY4dgGHftC4urn5UGRE1IwLPUAUKWmU=; b=MvVqF6OBkeoC7W7h08RMRuLfXBstRCnyDWGZZBg0/lIKMj0flbgH6EwhwJtk7aCiDM 1IdN6FD6z7YqYNJl0nsS9Y5vmcbmILfZVtwPpujLKOohSG1a745yCeZlawFZpdW63oHm dzN1wPJ4XLHYG39f3VsDzDDnJCvnOWxWRZjCi7Ags7+cJagoTbzAXYXUbmbZgiml89Mu wmOAi4C9EF9GI1u/p2gbQlNwnuThGkiAMFIJUSXXfsw5RrcGP/DWeojXfYG09DQgGx/5 audiKz/+aj4pnf2XwT9PtcDfXNqksim8x4fQsG2T465afPVdlT5eew5p4PByTC52zvBy 51sQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=hOZXrSo9cXF1wY4dgGHftC4urn5UGRE1IwLPUAUKWmU=; b=YWuARX6vShzWLmBHTnJJhxqde1peapoSZ1jslZU7FYlvMvgxnciP/sqoxsjWZ8bMxV Jsz9SAGgf/8pUVLeu6KqHurJ7YbH6jNNTt6NCpKn0BcyEFf52KmAMzsrYLFnYwAag7wz IUaxIyoX6a7jMQTqjwn3Zij4HSamN32sGDGdj7wrQQCtuvxGhYCMBeHUHlR7mSuPCuGj i6p9UGI0j7gicjlH3u++Whq8rvNh9/tPGtg9EEbX0wf8Ltj6lEkliCbbEzJBcstCxy03 7e5MhWB6vHm9fxT1s3xHiCjpz3mwUo7v/n9S7OlOp8TToBTD1hrBBlzbO3/iJLaUTq0r G0DA== X-Gm-Message-State: AOAM533BlW8i9wJwOjZmYh6KEML5/QXNiZ+8tKHnW7W7s3lfr9mrgzpE +s6jXPkvifH7q6kvN8qqxiwJq62JMNs= X-Received: by 2002:a17:90a:c293:: with SMTP id f19mr1350383pjt.91.1591146834107; Tue, 02 Jun 2020 18:13:54 -0700 (PDT) Received: from localhost.localdomain (174-21-143-238.tukw.qwest.net. [174.21.143.238]) by smtp.gmail.com with ESMTPSA id 3sm290067pfe.85.2020.06.02.18.13.52 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 02 Jun 2020 18:13:53 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v7 27/42] target/arm: Add helper_mte_check_zva Date: Tue, 2 Jun 2020 18:13:02 -0700 Message-Id: <20200603011317.473934-28-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200603011317.473934-1-richard.henderson@linaro.org> References: <20200603011317.473934-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::542; envelope-from=richard.henderson@linaro.org; helo=mail-pg1-x542.google.com X-detected-operating-system: by eggs.gnu.org: No matching host in p0f cache. That's all we know. X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: peter.maydell@linaro.org, qemu-arm@nongnu.org, steplong@quicinc.com Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" Use a special helper for DC_ZVA, rather than the more general mte_checkN. Leave the helper blank for now. Signed-off-by: Richard Henderson --- target/arm/helper-a64.h | 1 + target/arm/mte_helper.c | 106 +++++++++++++++++++++++++++++++++++++ target/arm/translate-a64.c | 16 +++++- 3 files changed, 122 insertions(+), 1 deletion(-) -- 2.25.1 Reviewed-by: Peter Maydell diff --git a/target/arm/helper-a64.h b/target/arm/helper-a64.h index 005af678c7..5b0b699a50 100644 --- a/target/arm/helper-a64.h +++ b/target/arm/helper-a64.h @@ -106,6 +106,7 @@ DEF_HELPER_FLAGS_2(xpacd, TCG_CALL_NO_RWG_SE, i64, env, i64) DEF_HELPER_FLAGS_3(mte_check1, TCG_CALL_NO_WG, i64, env, i32, i64) DEF_HELPER_FLAGS_3(mte_checkN, TCG_CALL_NO_WG, i64, env, i32, i64) +DEF_HELPER_FLAGS_3(mte_check_zva, TCG_CALL_NO_WG, i64, env, i32, i64) DEF_HELPER_FLAGS_3(irg, TCG_CALL_NO_RWG, i64, env, i64, i64) DEF_HELPER_FLAGS_4(addsubg, TCG_CALL_NO_RWG_SE, i64, env, i64, s32, i32) DEF_HELPER_FLAGS_3(ldg, TCG_CALL_NO_WG, i64, env, i64, i64) diff --git a/target/arm/mte_helper.c b/target/arm/mte_helper.c index c8a523ed4e..175da11e01 100644 --- a/target/arm/mte_helper.c +++ b/target/arm/mte_helper.c @@ -661,3 +661,109 @@ uint64_t HELPER(mte_checkN)(CPUARMState *env, uint32_t desc, uint64_t ptr) { return mte_checkN(env, desc, ptr, GETPC()); } + +/* + * Perform an MTE checked access for DC_ZVA. + */ +uint64_t HELPER(mte_check_zva)(CPUARMState *env, uint32_t desc, uint64_t ptr) +{ + uintptr_t ra = GETPC(); + int log2_dcz_bytes, log2_tag_bytes; + int mmu_idx, bit55; + intptr_t dcz_bytes, tag_bytes, i; + void *mem; + uint64_t ptr_tag, mem_tag, align_ptr; + + bit55 = extract64(ptr, 55, 1); + + /* If TBI is disabled, the access is unchecked, and ptr is not dirty. */ + if (unlikely(!tbi_check(desc, bit55))) { + return ptr; + } + + ptr_tag = allocation_tag_from_addr(ptr); + + if (tcma_check(desc, bit55, ptr_tag)) { + goto done; + } + + /* + * In arm_cpu_realizefn, we asserted that dcz > LOG2_TAG_GRANULE+1, + * i.e. 32 bytes, which is an unreasonably small dcz anyway, to make + * sure that we can access one complete tag byte here. + */ + log2_dcz_bytes = env_archcpu(env)->dcz_blocksize + 2; + log2_tag_bytes = log2_dcz_bytes - (LOG2_TAG_GRANULE + 1); + dcz_bytes = (intptr_t)1 << log2_dcz_bytes; + tag_bytes = (intptr_t)1 << log2_tag_bytes; + align_ptr = ptr & -dcz_bytes; + + /* + * Trap if accessing an invalid page. DC_ZVA requires that we supply + * the original pointer for an invalid page. But watchpoints require + * that we probe the actual space. So do both. + */ + mmu_idx = FIELD_EX32(desc, MTEDESC, MIDX); + (void) probe_write(env, ptr, 1, mmu_idx, ra); + mem = allocation_tag_mem(env, mmu_idx, align_ptr, MMU_DATA_STORE, + dcz_bytes, MMU_DATA_LOAD, tag_bytes, ra); + if (!mem) { + goto done; + } + + /* + * Unlike the reasoning for checkN, DC_ZVA is always aligned, and thus + * it is quite easy to perform all of the comparisons at once without + * any extra masking. + * + * The most common zva block size is 64; some of the thunderx cpus use + * a block size of 128. For user-only, aarch64_max_initfn will set the + * block size to 512. Fill out the other cases for future-proofing. + * + * In order to be able to find the first miscompare later, we want the + * tag bytes to be in little-endian order. + */ + switch (log2_tag_bytes) { + case 0: /* zva_blocksize 32 */ + mem_tag = *(uint8_t *)mem; + ptr_tag *= 0x11u; + break; + case 1: /* zva_blocksize 64 */ + mem_tag = cpu_to_le16(*(uint16_t *)mem); + ptr_tag *= 0x1111u; + break; + case 2: /* zva_blocksize 128 */ + mem_tag = cpu_to_le32(*(uint32_t *)mem); + ptr_tag *= 0x11111111u; + break; + case 3: /* zva_blocksize 256 */ + mem_tag = cpu_to_le64(*(uint64_t *)mem); + ptr_tag *= 0x1111111111111111ull; + break; + + default: /* zva_blocksize 512, 1024, 2048 */ + ptr_tag *= 0x1111111111111111ull; + i = 0; + do { + mem_tag = cpu_to_le64(*(uint64_t *)(mem + i)); + if (unlikely(mem_tag != ptr_tag)) { + goto fail; + } + i += 8; + align_ptr += 16 * TAG_GRANULE; + } while (i < tag_bytes); + goto done; + } + + if (likely(mem_tag == ptr_tag)) { + goto done; + } + + fail: + /* Locate the first nibble that differs. */ + i = ctz64(mem_tag ^ ptr_tag) >> 4; + mte_check_fail(env, mmu_idx, align_ptr + i * TAG_GRANULE, ra); + + done: + return useronly_clean_ptr(ptr); +} diff --git a/target/arm/translate-a64.c b/target/arm/translate-a64.c index f40eee5a14..c7deeca86a 100644 --- a/target/arm/translate-a64.c +++ b/target/arm/translate-a64.c @@ -1848,7 +1848,21 @@ static void handle_sys(DisasContext *s, uint32_t insn, bool isread, return; case ARM_CP_DC_ZVA: /* Writes clear the aligned block of memory which rt points into. */ - tcg_rt = clean_data_tbi(s, cpu_reg(s, rt)); + if (s->mte_active[0]) { + TCGv_i32 t_desc; + int desc = 0; + + desc = FIELD_DP32(desc, MTEDESC, MIDX, get_mem_index(s)); + desc = FIELD_DP32(desc, MTEDESC, TBI, s->tbid); + desc = FIELD_DP32(desc, MTEDESC, TCMA, s->tcma); + t_desc = tcg_const_i32(desc); + + tcg_rt = new_tmp_a64(s); + gen_helper_mte_check_zva(tcg_rt, cpu_env, t_desc, cpu_reg(s, rt)); + tcg_temp_free_i32(t_desc); + } else { + tcg_rt = clean_data_tbi(s, cpu_reg(s, rt)); + } gen_helper_dc_zva(cpu_env, tcg_rt); return; default: From patchwork Wed Jun 3 01:13:03 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 187423 Delivered-To: patch@linaro.org Received: by 2002:a92:cf06:0:0:0:0:0 with SMTP id c6csp75895ilo; Tue, 2 Jun 2020 18:24:30 -0700 (PDT) X-Google-Smtp-Source: ABdhPJyTn1UK8u8SqUUIpayWPehAMd2ScL8yNw+x7R59hshdVnv2cKT2Yr9JpXIPLeFVIbkDFUjp X-Received: by 2002:a25:d753:: with SMTP id o80mr15019547ybg.423.1591147470359; Tue, 02 Jun 2020 18:24:30 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1591147470; cv=none; d=google.com; s=arc-20160816; b=F5EJ96hOFMSmJcgc20mmcR20WOHiKTjZi7wUoEPIxTbJv7bikTf/LmeU7cgKjlZFnk HAV5oiGwI8qvSsy/EzKo3D4qM5eWCTnFywB/kwzYSDOJ2tCCVLcW+ilrxkQSXceLWYcE g1cVWnNzUJAof5dQJYrS4ctxKp++IFBdCPovhPj8qaDl1Zv2+oK3BeTAxbEuJpE+xMnr gLQ+ALV0FIxn6ZBCSza6vjo7nlSX2QS9nadNfhyfVKeZXpAnz+mlNFGxTynCqLhYbisu jAdYZebsoHKyN2j34MS6NC/PU0NXu+tyjLDPuZ/g75mlGiY5TeXPRFXU7O6xRTFuAlbb gibg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=p2/SXxqi/mpzPKEmHvCWYtwtCXux2aJOwAbCs1scM98=; b=suJC6g7X19OpeLJUKWrOulnPJIhSg5eqmEcNeAWRCf32NeKrrYvOinOyyIVOUsr2VF kMFXi1wnfvjce27+ZnDQEmAazO3pGHSfoY9krLqvfdavWdQ4PVPZC+CeQxY8xvr3VS8z H8kmdOZRrrsMMlgLZOd5YnNJEqvN+gLMmcFol6yCixavrLE6+mwPCT0fx40rWw0AY3OA 1b9MrpKRsoPAfZhB+JCjKR3OVjQsk8GVdUjp95aY3wCrehyxRtRfhpwerkNmwvX85Qvc KrF5UNAAra5NusxasSFqqCF/knfS4fJ+eGqHEJeBpZoHVHmJR/YG3BSCrOxMpvVMKkda 3jbQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=i4zYhcGz; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id x18si320986ybj.303.2020.06.02.18.24.30 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 02 Jun 2020 18:24:30 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=i4zYhcGz; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:38870 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jgI8r-0007Dz-Ms for patch@linaro.org; Tue, 02 Jun 2020 21:24:29 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:48898) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jgHyg-0004WE-95 for qemu-devel@nongnu.org; Tue, 02 Jun 2020 21:13:58 -0400 Received: from mail-pf1-x443.google.com ([2607:f8b0:4864:20::443]:46897) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1jgHyf-0003eD-3v for qemu-devel@nongnu.org; Tue, 02 Jun 2020 21:13:57 -0400 Received: by mail-pf1-x443.google.com with SMTP id b16so396227pfi.13 for ; Tue, 02 Jun 2020 18:13:56 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=p2/SXxqi/mpzPKEmHvCWYtwtCXux2aJOwAbCs1scM98=; b=i4zYhcGzr8zq6qOUKRVqD5zmliiA3slbkyILaQ+Lj4p75hh6PFbWc/h0hRPICTdBby 4KoKCIHU5omb7UW3jhU0lej9yNTD1GUyVWj0DauSp8B5KRnR35TyY5AFVvRk2sHLSneR 3gNv6nv5YSvz6v6shT98Dv6r6SN5EI+a9sQgdrNFKnimOzr43qmroLXPrInBpwLwii4m c2bXNPu5q2Aq5djP4L9ezhnsF1tm75T9VZksuq1ZnTlDTj3OPxn18R1Jl6f6YxKEIc4c inDMJiUyIHkXTMxAgKZOb5sMjyOf58MH5teW9RLcRRGRikU63W9mVp9frpgoi1bOf4Dc nWIA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=p2/SXxqi/mpzPKEmHvCWYtwtCXux2aJOwAbCs1scM98=; b=Orr3uAMX835FXKOoXBzGZOkdwow6zo3jwzxmB9XyLtpUZFwuub7PCkVFD8WaawwC/m JUonw81qd0kz12GrlLUB4b8ZZIOa9w9tXCI5gZE5iuxdwRypIuskLPFGMt8lijENknwe ga4OtItED9YFAcQlusuZ5Ici+myy25NDZeR/0xFlVHvelyd5V7qaWYtGLJPlaOlnJ1hM Vt6Qycp4ew5YX2sMsTllc7+rGPqLzmsH52qNFaUYoQ2IK1CdabogrJZkeELaCSgOgJO8 KZFVCdkb3bQFqaV6WEXs09Y7ELrbRCq9+pAWJgAfgb+E/b5dOaz1d+DgTlnPc/T8yhIw +LJA== X-Gm-Message-State: AOAM533APVpm4o8oy6v4EGe6yBoUO1j01Pc1yrswtykNLuaQ+D+FybYX dCOy91oRTXHjI/VLG0WLvMpUMKfJzA4= X-Received: by 2002:a17:90b:234c:: with SMTP id ms12mr2486958pjb.164.1591146835254; Tue, 02 Jun 2020 18:13:55 -0700 (PDT) Received: from localhost.localdomain (174-21-143-238.tukw.qwest.net. [174.21.143.238]) by smtp.gmail.com with ESMTPSA id 3sm290067pfe.85.2020.06.02.18.13.54 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 02 Jun 2020 18:13:54 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v7 28/42] target/arm: Use mte_checkN for sve unpredicated loads Date: Tue, 2 Jun 2020 18:13:03 -0700 Message-Id: <20200603011317.473934-29-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200603011317.473934-1-richard.henderson@linaro.org> References: <20200603011317.473934-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::443; envelope-from=richard.henderson@linaro.org; helo=mail-pf1-x443.google.com X-detected-operating-system: by eggs.gnu.org: No matching host in p0f cache. That's all we know. X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: peter.maydell@linaro.org, qemu-arm@nongnu.org, steplong@quicinc.com Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" Signed-off-by: Richard Henderson --- target/arm/helper-sve.h | 2 + target/arm/sve_helper.c | 70 +++++++++++++++++++++++++++- target/arm/translate-sve.c | 93 ++++++++++++++++---------------------- 3 files changed, 109 insertions(+), 56 deletions(-) -- 2.25.1 diff --git a/target/arm/helper-sve.h b/target/arm/helper-sve.h index 7a200755ac..550ae7f7e6 100644 --- a/target/arm/helper-sve.h +++ b/target/arm/helper-sve.h @@ -1138,6 +1138,8 @@ DEF_HELPER_FLAGS_5(sve_ftmad_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_5(sve_ftmad_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_5(sve_ftmad_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve_ldr, TCG_CALL_NO_WG, void, env, ptr, tl, int) + DEF_HELPER_FLAGS_4(sve_ld1bb_r, TCG_CALL_NO_WG, void, env, ptr, tl, i32) DEF_HELPER_FLAGS_4(sve_ld2bb_r, TCG_CALL_NO_WG, void, env, ptr, tl, i32) DEF_HELPER_FLAGS_4(sve_ld3bb_r, TCG_CALL_NO_WG, void, env, ptr, tl, i32) diff --git a/target/arm/sve_helper.c b/target/arm/sve_helper.c index e590db6637..04340cbe0c 100644 --- a/target/arm/sve_helper.c +++ b/target/arm/sve_helper.c @@ -3935,7 +3935,7 @@ void HELPER(sve_fcmla_zpzzz_d)(void *vd, void *vn, void *vm, void *va, */ /* - * Load one element into @vd + @reg_off from @host. + * Load one element into @vd + @reg_off, from @host. * The controlling predicate is known to be true. */ typedef void sve_ldst1_host_fn(void *vd, intptr_t reg_off, void *host); @@ -4129,6 +4129,74 @@ static bool sve_probe_page(SVEHostPage *info, bool nofault, return true; } +/* + * Load contiguous data, unpredicated. + * + * Note that unpredicated load/store of vector/predicate registers + * are defined as a stream of bytes, which equates to little-endian + * operations on larger quantities. + * + * Note any MTE check is already handled. + */ + +void HELPER(sve_ldr)(CPUARMState *env, void *vd, target_ulong addr, int size) +{ + int mmu_idx = cpu_mmu_index(env, false); + int in_page = -((int)addr | TARGET_PAGE_MASK); + uintptr_t ra = GETPC(); + uint64_t val; + int i; + + /* Small loads are expanded inline. */ + tcg_debug_assert(size > 2 * 8); + + /* Bulk copy the data from memory to the register. */ + if (likely(size <= in_page)) { + void *host = probe_read(env, addr, size, mmu_idx, ra); + + if (unlikely(!host)) { + goto mmio; + } + memcpy(vd, host, size); + } else { + void *h1 = probe_read(env, addr, in_page, mmu_idx, ra); + void *h2 = probe_read(env, addr + in_page, size - in_page, mmu_idx, ra); + + if (unlikely(!h1 || !h2)) { + goto mmio; + } + memcpy(vd, h1, in_page); + memcpy(vd + in_page, h2, size - in_page); + } + + /* Predicate load length may be any multiple of 2; ensure high bits 0. */ + if (unlikely(size & 7)) { + memset(vd + size, 0, 8 - (size & 7)); + } + + /* + * The memcpy and memset above kept the bytes in memory order. + * The in-register format has uint64_t in host order, so for + * big-endian host we need to bswap. + */ + for (i = 0; i < size; i += 8) { + le64_to_cpus(vd + i); + } + return; + + mmio: + for (i = 0; i + 8 <= size; i += 8) { + val = cpu_ldq_le_data_ra(env, addr + i, ra); + *(uint64_t *)(vd + i) = val; + } + + /* Predicate load length may be any multiple of 2. */ + if (unlikely(i != size)) { + val = cpu_ldq_le_data_ra(env, addr + size - 8, ra); + val >>= (size - i) * 8; + *(uint64_t *)(vd + i + 8) = val; + } +} /* * Analyse contiguous data, protected by a governing predicate. diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index ac7b3119e5..35b979fa13 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -4332,8 +4332,13 @@ static bool trans_UCVTF_dd(DisasContext *s, arg_rpr_esz *a) *** SVE Memory - 32-bit Gather and Unsized Contiguous Group */ -/* Subroutine loading a vector register at VOFS of LEN bytes. +/* + * Subroutine loading a vector register at VOFS of LEN bytes. * The load should begin at the address Rn + IMM. + * + * Note that unpredicated load/store of vector/predicate registers + * are defined as a stream of bytes, which equates to little-endian + * operations on larger quantities. */ static void do_ldr(DisasContext *s, uint32_t vofs, int len, int rn, int imm) @@ -4342,81 +4347,59 @@ static void do_ldr(DisasContext *s, uint32_t vofs, int len, int rn, int imm) int len_remain = len % 8; int nparts = len / 8 + ctpop8(len_remain); int midx = get_mem_index(s); - TCGv_i64 addr, t0, t1; + TCGv_i64 dirty_addr, clean_addr, t0, t1; + int i; - addr = tcg_temp_new_i64(); - t0 = tcg_temp_new_i64(); + dirty_addr = read_cpu_reg_sp(s, rn, true); + tcg_gen_addi_i64(dirty_addr, dirty_addr, imm); - /* Note that unpredicated load/store of vector/predicate registers - * are defined as a stream of bytes, which equates to little-endian - * operations on larger quantities. There is no nice way to force - * a little-endian load for aarch64_be-linux-user out of line. - * - * Attempt to keep code expansion to a minimum by limiting the - * amount of unrolling done. - */ - if (nparts <= 4) { - int i; + clean_addr = gen_mte_checkN(s, dirty_addr, false, rn != 31, len, MO_8); - for (i = 0; i < len_align; i += 8) { - tcg_gen_addi_i64(addr, cpu_reg_sp(s, rn), imm + i); - tcg_gen_qemu_ld_i64(t0, addr, midx, MO_LEQ); - tcg_gen_st_i64(t0, cpu_env, vofs + i); - } - } else { - TCGLabel *loop = gen_new_label(); - TCGv_ptr tp, i = tcg_const_local_ptr(0); + /* Limit tcg code expansion by doing large loads out of line. */ + if (nparts > 4) { + TCGv_ptr t_rd = tcg_temp_new_ptr(); + TCGv_i32 t_len = tcg_const_i32(len); - gen_set_label(loop); - - /* Minimize the number of local temps that must be re-read from - * the stack each iteration. Instead, re-compute values other - * than the loop counter. - */ - tp = tcg_temp_new_ptr(); - tcg_gen_addi_ptr(tp, i, imm); - tcg_gen_extu_ptr_i64(addr, tp); - tcg_gen_add_i64(addr, addr, cpu_reg_sp(s, rn)); - - tcg_gen_qemu_ld_i64(t0, addr, midx, MO_LEQ); - - tcg_gen_add_ptr(tp, cpu_env, i); - tcg_gen_addi_ptr(i, i, 8); - tcg_gen_st_i64(t0, tp, vofs); - tcg_temp_free_ptr(tp); - - tcg_gen_brcondi_ptr(TCG_COND_LTU, i, len_align, loop); - tcg_temp_free_ptr(i); + tcg_gen_addi_ptr(t_rd, cpu_env, vofs); + gen_helper_sve_ldr(cpu_env, t_rd, clean_addr, t_len); + tcg_temp_free_ptr(t_rd); + tcg_temp_free_i32(t_len); + return; } - /* Predicate register loads can be any multiple of 2. - * Note that we still store the entire 64-bit unit into cpu_env. + t0 = tcg_temp_new_i64(); + for (i = 0; i < len_align; i += 8) { + tcg_gen_qemu_ld_i64(t0, clean_addr, midx, MO_LEQ); + tcg_gen_st_i64(t0, cpu_env, vofs + i); + tcg_gen_addi_i64(clean_addr, clean_addr, 8); + } + + /* + * Predicate register loads can be any multiple of 2. + * Note that we still store the entire 64-bit unit into cpu_env + * so that the high bits are zeroed. */ if (len_remain) { - tcg_gen_addi_i64(addr, cpu_reg_sp(s, rn), imm + len_align); - switch (len_remain) { case 2: - case 4: - case 8: - tcg_gen_qemu_ld_i64(t0, addr, midx, MO_LE | ctz32(len_remain)); + tcg_gen_qemu_ld_i64(t0, clean_addr, midx, MO_LEUW); + break; + case 4: + tcg_gen_qemu_ld_i64(t0, clean_addr, midx, MO_LEUL); break; - case 6: t1 = tcg_temp_new_i64(); - tcg_gen_qemu_ld_i64(t0, addr, midx, MO_LEUL); - tcg_gen_addi_i64(addr, addr, 4); - tcg_gen_qemu_ld_i64(t1, addr, midx, MO_LEUW); + tcg_gen_qemu_ld_i64(t0, clean_addr, midx, MO_LEUL); + tcg_gen_addi_i64(clean_addr, clean_addr, 4); + tcg_gen_qemu_ld_i64(t1, clean_addr, midx, MO_LEUW); tcg_gen_deposit_i64(t0, t0, t1, 32, 32); tcg_temp_free_i64(t1); break; - default: g_assert_not_reached(); } tcg_gen_st_i64(t0, cpu_env, vofs + len_align); } - tcg_temp_free_i64(addr); tcg_temp_free_i64(t0); } From patchwork Wed Jun 3 01:13:04 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 187425 Delivered-To: patch@linaro.org Received: by 2002:a92:cf06:0:0:0:0:0 with SMTP id c6csp76902ilo; Tue, 2 Jun 2020 18:26:17 -0700 (PDT) X-Google-Smtp-Source: ABdhPJyLsMgn00qcVqDwhPSkACfGJf+/TFVvExvnO69NgC2OIihUiUM6NYN34wF1cXqgj232T7DX X-Received: by 2002:a25:244a:: with SMTP id k71mr52775162ybk.143.1591147577869; Tue, 02 Jun 2020 18:26:17 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1591147577; cv=none; d=google.com; s=arc-20160816; b=V/49WOywidpLyMTOli8MIMHDZgpZFvf+U9lGHtK6pSih+B1I4FkZcKzFcV/1B5e5RH qJUNsmDMhsbQHANoGXS+tWNVAl19YB/pzIdnsa7CTfnsTViDoKTwKxNckp+7fhyNS/pC T1I2YaEjipJWC2d2bhjfgj3goUwUSr+4ciEmRayts1oeuEHok7jZm++xodOkLCZ0wZqJ kkGX9TYc139M7WthpKrxVRIBAJWRJKuz4+rG3KPDsUqP93Awb1Kq06ixFA5zHabwck1M YuTOXSMKBgC/voGYNxqwxeusaeLiHXQQPEhr/Vm1lCJqlD1EfXk6q3lJPBAmTYGSN3Kw fs+Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=uyR9T9oGRaQ7j6mqreds6AjvFFxk3QgOhv9SfFUw7cA=; b=tTlCclomXgQDJqQwrjxUTFVvOyDi1GZZLy4iV1B5LoAybHPFBW1tzYaZtlX0ki48sN VjDWNHRFWLzo8Iqta1dVHc7FrBC2qIKiBF8/2PsEMMnKnQ7yXPRdfs5Te5ImYzF1OgCO lRMGcjktCpWCWCtWPsDoiq8OJ6oyFYG6RT6Mxjq6mQmC8OlGTNlEOcuRtdhxWOdHqrgN pN5ydobSZXGFk8pxg81C9TUSRkZVl44/fPkPVqFMXIAvOazDpZvLMQRio1LdMAA/tblm iw4L39XK3kB8OY4fDxS6sjoiSVnWCmfy4+yZMTdEfdpuiGyAiQhO7g7BzITde+9BTzEU KBjw== ARC-Authentication-Results: i=1; mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b="wsGXO/XY"; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id c4si350191ybn.148.2020.06.02.18.26.17 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 02 Jun 2020 18:26:17 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b="wsGXO/XY"; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:47648 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jgIAb-0002PN-AX for patch@linaro.org; Tue, 02 Jun 2020 21:26:17 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:48908) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jgHyh-0004Zk-9j for qemu-devel@nongnu.org; Tue, 02 Jun 2020 21:13:59 -0400 Received: from mail-pg1-x543.google.com ([2607:f8b0:4864:20::543]:43258) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1jgHyg-0003eV-Bc for qemu-devel@nongnu.org; Tue, 02 Jun 2020 21:13:58 -0400 Received: by mail-pg1-x543.google.com with SMTP id 185so529575pgb.10 for ; Tue, 02 Jun 2020 18:13:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=uyR9T9oGRaQ7j6mqreds6AjvFFxk3QgOhv9SfFUw7cA=; b=wsGXO/XY41ZrbUG+OMMBSYVNXWVkXfimq3NIwi4b97AUZJblU8SLzcNJSGgGZnBPiD FxD6ecT+9Z8vOrXsGYHhwRd/YnKHc62hOIe+ygzjh5uLxmXwsx42hflX1t5uLi8X9JKH cCtMKPmnhAcDihnGs6DK5CfsAfFLDHK4HmuGFYMCAa3LDmSy05iLJ+8Kc0sK+zG61gvw sLqbtnu01tL4MumIvExJ3bAgpe7Rn87Pm+k9uJPwYiSfDVAlH29agAlnf7tpexb+QiLs 8GTtwhJwFCEL8M1HtCpGHvGiecRSJcjcJuEGFn7L/aDACom6YU9CF8kihAwX4D5tCRH4 4ZtQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=uyR9T9oGRaQ7j6mqreds6AjvFFxk3QgOhv9SfFUw7cA=; b=uKR1EusC2UAmWl28aHwiyCI7BKCB3TKWGzodMvaiqb0UZTH9VPP+tPfrH4BJyDu2uN SpdnhkPWY3pA+cdBegwtgR6yhpImnsBfgee6kEX+88BXGokW2NUqPPXRZ+OOEEl3Kvh6 QGKpSTFxyIRH4Yr34LNCZs0Tnf4OxioHkuPB7XUWvT7pIuMHgNsJUDmEZtSiBpjLl4Zj J16fv6EQsngIWmE7iRNkJ1Ghe2gWve0+EWO632N+AAPouNwKCfg59XUELzQ34Bp3amzK kNwpELDRSbd19CqOW+hy3sjfco8xyC5xy690ZGexyHjahEkb1Qa383Sdq0k2QyYUCndU O/9g== X-Gm-Message-State: AOAM533hsequ6dbGHVriqqUzv11i9O09MdDJJ7k8Wp/ro6jbh0cnQEp6 n7I0BLmT85822IAwCCtXxLuJdrgWvQE= X-Received: by 2002:a17:90a:5a07:: with SMTP id b7mr2247945pjd.130.1591146836561; Tue, 02 Jun 2020 18:13:56 -0700 (PDT) Received: from localhost.localdomain (174-21-143-238.tukw.qwest.net. [174.21.143.238]) by smtp.gmail.com with ESMTPSA id 3sm290067pfe.85.2020.06.02.18.13.55 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 02 Jun 2020 18:13:55 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v7 29/42] target/arm: Use mte_checkN for sve unpredicated stores Date: Tue, 2 Jun 2020 18:13:04 -0700 Message-Id: <20200603011317.473934-30-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200603011317.473934-1-richard.henderson@linaro.org> References: <20200603011317.473934-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::543; envelope-from=richard.henderson@linaro.org; helo=mail-pg1-x543.google.com X-detected-operating-system: by eggs.gnu.org: No matching host in p0f cache. That's all we know. X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: peter.maydell@linaro.org, qemu-arm@nongnu.org, steplong@quicinc.com Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" Signed-off-by: Richard Henderson --- target/arm/helper-sve.h | 1 + target/arm/sve_helper.c | 63 ++++++++++++++++++++++++++- target/arm/translate-sve.c | 88 ++++++++++++++------------------------ 3 files changed, 94 insertions(+), 58 deletions(-) -- 2.25.1 diff --git a/target/arm/helper-sve.h b/target/arm/helper-sve.h index 550ae7f7e6..197cd72999 100644 --- a/target/arm/helper-sve.h +++ b/target/arm/helper-sve.h @@ -1139,6 +1139,7 @@ DEF_HELPER_FLAGS_5(sve_ftmad_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_5(sve_ftmad_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(sve_ldr, TCG_CALL_NO_WG, void, env, ptr, tl, int) +DEF_HELPER_FLAGS_4(sve_str, TCG_CALL_NO_WG, void, env, ptr, tl, int) DEF_HELPER_FLAGS_4(sve_ld1bb_r, TCG_CALL_NO_WG, void, env, ptr, tl, i32) DEF_HELPER_FLAGS_4(sve_ld2bb_r, TCG_CALL_NO_WG, void, env, ptr, tl, i32) diff --git a/target/arm/sve_helper.c b/target/arm/sve_helper.c index 04340cbe0c..507fba37c9 100644 --- a/target/arm/sve_helper.c +++ b/target/arm/sve_helper.c @@ -4130,7 +4130,7 @@ static bool sve_probe_page(SVEHostPage *info, bool nofault, } /* - * Load contiguous data, unpredicated. + * Load/store contiguous data, unpredicated. * * Note that unpredicated load/store of vector/predicate registers * are defined as a stream of bytes, which equates to little-endian @@ -4198,6 +4198,67 @@ void HELPER(sve_ldr)(CPUARMState *env, void *vd, target_ulong addr, int size) } } +void HELPER(sve_str)(CPUARMState *env, void *vd, target_ulong addr, int size) +{ + int mem_idx = cpu_mmu_index(env, false); + int in_page = -((int)addr | TARGET_PAGE_MASK); + uintptr_t ra = GETPC(); + uint64_t val; + void *host; + int i; + + /* Small stores are expanded inline. */ + tcg_debug_assert(size > 2 * 8); + + if (likely(size <= in_page)) { + host = probe_write(env, addr, size, mem_idx, ra); + if (likely(host != NULL)) { + for (i = 0; i + 8 <= size; i += 8) { + stq_le_p(host + i, *(uint64_t *)(vd + i)); + } + + /* Predicate load length may be any multiple of 2. */ + if (unlikely(i != size)) { + val = *(uint64_t *)(vd + i); + if (size & 4) { + stl_le_p(host + i, val); + i += 4; + val >>= 32; + } + if (size & 2) { + stw_le_p(host + i, val); + } + } + return; + } + } else { + (void)probe_write(env, addr, in_page, mem_idx, ra); + (void)probe_write(env, addr + in_page, size - in_page, mem_idx, ra); + } + + /* + * Note there is no endian-specific target store function, so to handle + * aarch64_be-linux-user we need to bswap the big-endian store. + */ + for (i = 0; i + 8 <= size; i += 8) { + val = *(uint64_t *)(vd + i); + cpu_stq_le_data_ra(env, addr + i, val, ra); + } + + /* Predicate load length may be any multiple of 2. */ + if (unlikely(i != size)) { + val = *(uint64_t *)(vd + i); + if (size & 4) { + cpu_stl_le_data_ra(env, addr + i, val, ra); + i += 4; + val >>= 32; + } + if (size & 2) { + cpu_stw_le_data_ra(env, addr + i, val, ra); + } + } +} + /* * Analyse contiguous data, protected by a governing predicate. */ diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index 35b979fa13..e515646db2 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -4410,78 +4410,52 @@ static void do_str(DisasContext *s, uint32_t vofs, int len, int rn, int imm) int len_remain = len % 8; int nparts = len / 8 + ctpop8(len_remain); int midx = get_mem_index(s); - TCGv_i64 addr, t0; + TCGv_i64 dirty_addr, clean_addr, t0; + int i; + + dirty_addr = read_cpu_reg_sp(s, rn, true); + tcg_gen_addi_i64(dirty_addr, dirty_addr, imm); + + clean_addr = gen_mte_checkN(s, dirty_addr, true, rn != 31, len, MO_8); + + /* Limit tcg code expansion by doing large loads out of line. */ + if (nparts > 4) { + TCGv_ptr t_rd = tcg_temp_new_ptr(); + TCGv_i32 t_len = tcg_const_i32(len); + + tcg_gen_addi_ptr(t_rd, cpu_env, vofs); + gen_helper_sve_str(cpu_env, t_rd, clean_addr, t_len); + tcg_temp_free_ptr(t_rd); + tcg_temp_free_i32(t_len); + return; + } - addr = tcg_temp_new_i64(); t0 = tcg_temp_new_i64(); - - /* Note that unpredicated load/store of vector/predicate registers - * are defined as a stream of bytes, which equates to little-endian - * operations on larger quantities. There is no nice way to force - * a little-endian store for aarch64_be-linux-user out of line. - * - * Attempt to keep code expansion to a minimum by limiting the - * amount of unrolling done. - */ - if (nparts <= 4) { - int i; - - for (i = 0; i < len_align; i += 8) { - tcg_gen_ld_i64(t0, cpu_env, vofs + i); - tcg_gen_addi_i64(addr, cpu_reg_sp(s, rn), imm + i); - tcg_gen_qemu_st_i64(t0, addr, midx, MO_LEQ); - } - } else { - TCGLabel *loop = gen_new_label(); - TCGv_ptr t2, i = tcg_const_local_ptr(0); - - gen_set_label(loop); - - t2 = tcg_temp_new_ptr(); - tcg_gen_add_ptr(t2, cpu_env, i); - tcg_gen_ld_i64(t0, t2, vofs); - - /* Minimize the number of local temps that must be re-read from - * the stack each iteration. Instead, re-compute values other - * than the loop counter. - */ - tcg_gen_addi_ptr(t2, i, imm); - tcg_gen_extu_ptr_i64(addr, t2); - tcg_gen_add_i64(addr, addr, cpu_reg_sp(s, rn)); - tcg_temp_free_ptr(t2); - - tcg_gen_qemu_st_i64(t0, addr, midx, MO_LEQ); - - tcg_gen_addi_ptr(i, i, 8); - - tcg_gen_brcondi_ptr(TCG_COND_LTU, i, len_align, loop); - tcg_temp_free_ptr(i); + for (i = 0; i < len_align; i += 8) { + tcg_gen_ld_i64(t0, cpu_env, vofs + i); + tcg_gen_qemu_st_i64(t0, clean_addr, midx, MO_LEQ); + tcg_gen_addi_i64(clean_addr, clean_addr, 8); } /* Predicate register stores can be any multiple of 2. */ if (len_remain) { tcg_gen_ld_i64(t0, cpu_env, vofs + len_align); - tcg_gen_addi_i64(addr, cpu_reg_sp(s, rn), imm + len_align); - switch (len_remain) { - case 2: - case 4: - case 8: - tcg_gen_qemu_st_i64(t0, addr, midx, MO_LE | ctz32(len_remain)); - break; - case 6: - tcg_gen_qemu_st_i64(t0, addr, midx, MO_LEUL); - tcg_gen_addi_i64(addr, addr, 4); + tcg_gen_qemu_st_i64(t0, clean_addr, midx, MO_LEUL); + tcg_gen_addi_i64(clean_addr, clean_addr, 4); tcg_gen_shri_i64(t0, t0, 32); - tcg_gen_qemu_st_i64(t0, addr, midx, MO_LEUW); + /* fall through */ + case 2: + tcg_gen_qemu_st_i64(t0, clean_addr, midx, MO_LEUW); + break; + case 4: + tcg_gen_qemu_st_i64(t0, clean_addr, midx, MO_LEUL); break; - default: g_assert_not_reached(); } } - tcg_temp_free_i64(addr); tcg_temp_free_i64(t0); } From patchwork Wed Jun 3 01:13:05 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 187428 Delivered-To: patch@linaro.org Received: by 2002:a92:cf06:0:0:0:0:0 with SMTP id c6csp77588ilo; Tue, 2 Jun 2020 18:27:28 -0700 (PDT) X-Google-Smtp-Source: ABdhPJxADn6XHTwx09CoMZmVbuFIyVmNQk62e55SgyrSFfr0fQCbJ+HoLrXs6g5r+qCOsNTNCB5w X-Received: by 2002:a5b:541:: with SMTP id r1mr45610539ybp.381.1591147648319; Tue, 02 Jun 2020 18:27:28 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1591147648; cv=none; d=google.com; s=arc-20160816; b=t9mxWTz1tKTUIRm/ek6batr6K7p33ioafdm7CtnNmz7DRFM8Ah5oEGlwulp/V+0UCJ ojJq4bGS7LiAzMlgqZ25+iQv4MKWgYGGDfSIYvROJ702YRpJytMciDeCKckamy7pwhiT 3Ua9G8Mnl7BnOuxU5zhV5hSRpfI0qSwNvyDGVzyAhHVPL2YltqaKwpI4JquZfkQ4mQ58 Wh/3U0He60I4M8RATs0RsbH9YjraR41HL1xa+PASnywTEHs0Kd8mlOMqcQf1ds8lX6OO QGc0pyOFbOBzt4JlvLT4tNB5RccOOcFPLV4CbXiXdIIIl2Fg7gCVJMLXLhdcZ9Ef49qs XGKw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=pWh7eNrZJvFxR8a07fRLkGAKBH4kVqj8pF+Brk+xMxs=; b=bt8/pd5dpN9OGt93uiWayuz2TXpukFUPfhReHexnfr3u4EwDNTv9Goa2aqI+8RfiiY lwqQvpgbt/VkEZlAAAPp6DxpQQ+Caqp2JaxVgV/wkenKaHS7LQCIRnxgRTu6FWkFOLSt r/wSZTezFOyejt7LyfjSUFltRexxsIu3Jvap1DwB4u8nKMkLCklao96/gmxxQDMj98bB i/vVc4VJzI2EuqeDvZVkXwIUlrH2vMfbS0RNwTrXefyfHdZHozoGC3ldxtPCpUZrn+KC shT3AsinbyeXlSH8UkAjMQlw+4xbGMTbbY6tdnVLqc6WYciP5Ua4lmXyQXlkGH1oLZbv S/SQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=rScdcdAj; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id d127si338270ybh.439.2020.06.02.18.27.28 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 02 Jun 2020 18:27:28 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=rScdcdAj; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:50846 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jgIBj-0003nL-RJ for patch@linaro.org; Tue, 02 Jun 2020 21:27:27 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:48920) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jgHyj-0004eq-0v for qemu-devel@nongnu.org; Tue, 02 Jun 2020 21:14:01 -0400 Received: from mail-pg1-x541.google.com ([2607:f8b0:4864:20::541]:35460) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1jgHyi-0003er-4F for qemu-devel@nongnu.org; Tue, 02 Jun 2020 21:14:00 -0400 Received: by mail-pg1-x541.google.com with SMTP id o6so579821pgh.2 for ; Tue, 02 Jun 2020 18:13:59 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=pWh7eNrZJvFxR8a07fRLkGAKBH4kVqj8pF+Brk+xMxs=; b=rScdcdAj3DugLGCAQKLiAg8MS+a/2tstaPXr9iJGTKrRy1vDFsJ04ojQ6+lkyBQ7wa ZBKhT4THswXAuaibp2L+m8hr10x1bdCXQBZw5by3Fdy5ek5u/X5rUTAB+H4k9G+Yd6u8 9xYXYrPEVPxTh3k0MAjSiACMVQE6UGxiods1lLys9fxWwfZGL8H99WTYtUR210/RansT aGEQoZmRTaP+7wtYJjJ8M7rHd5GhTtbfx+tdV6+HhytSNYCbTu0DfkrEreMd8d51nznN h+ZqlGvC701U1MkQ7eZisXJ1zK1vXyap0aw78joxQyszDUGdTUFurICYIcEzsjzyMdr2 RYGg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=pWh7eNrZJvFxR8a07fRLkGAKBH4kVqj8pF+Brk+xMxs=; b=fYoW1EjsGgLMTvkpL0bkpMYOiZvEW5J8JGQzetfkL0xxH96h4JNeGIqYwc4nZVHVrJ vy3YsqfgPidOWTTNf5jET57NcNgNPMOulX2vlQjlIYeBQLb95PYDEdaXfh/2+YvSGnFU ebqDRhr7kwgtmlq916pX3HnhisiLKUCvTTFL2iMnXgAW3h2NWdMpmQVXlohRpUXVIupn VjL8AgYbA6WAjM7HLc/dJVEH+ohEz/2NCNwfNkI5j5RrupT6iEIbKrcMY6rycguJTtGs +6HsCbKAhsz/z+3jwQ+7fDofl7AtRvWAbUT+xmw5DCpMlXrv6DcbScs6Ka4IweanlhKY NiaA== X-Gm-Message-State: AOAM530liNeRZO+zNBFxcifoehnY8AtRs6OryHY0VVv0Od6zAwCfcKCE g8Q9SeJf9iWQjstBGGPoFBsO2X9ydLE= X-Received: by 2002:a63:a84d:: with SMTP id i13mr25594284pgp.342.1591146838353; Tue, 02 Jun 2020 18:13:58 -0700 (PDT) Received: from localhost.localdomain (174-21-143-238.tukw.qwest.net. [174.21.143.238]) by smtp.gmail.com with ESMTPSA id 3sm290067pfe.85.2020.06.02.18.13.57 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 02 Jun 2020 18:13:57 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v7 30/42] target/arm: Use mte_check1 for sve LD1R Date: Tue, 2 Jun 2020 18:13:05 -0700 Message-Id: <20200603011317.473934-31-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200603011317.473934-1-richard.henderson@linaro.org> References: <20200603011317.473934-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::541; envelope-from=richard.henderson@linaro.org; helo=mail-pg1-x541.google.com X-detected-operating-system: by eggs.gnu.org: No matching host in p0f cache. That's all we know. X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: peter.maydell@linaro.org, qemu-arm@nongnu.org, steplong@quicinc.com Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" Signed-off-by: Richard Henderson --- target/arm/translate-sve.c | 14 ++++++++------ 1 file changed, 8 insertions(+), 6 deletions(-) -- 2.25.1 diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index e515646db2..4b3b879815 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -4830,16 +4830,16 @@ static bool trans_LD1RQ_zpri(DisasContext *s, arg_rpri_load *a) /* Load and broadcast element. */ static bool trans_LD1R_zpri(DisasContext *s, arg_rpri_load *a) { - if (!sve_access_check(s)) { - return true; - } - unsigned vsz = vec_full_reg_size(s); unsigned psz = pred_full_reg_size(s); unsigned esz = dtype_esz[a->dtype]; unsigned msz = dtype_msz(a->dtype); TCGLabel *over = gen_new_label(); - TCGv_i64 temp; + TCGv_i64 temp, clean_addr; + + if (!sve_access_check(s)) { + return true; + } /* If the guarding predicate has no bits set, no load occurs. */ if (psz <= 8) { @@ -4862,7 +4862,9 @@ static bool trans_LD1R_zpri(DisasContext *s, arg_rpri_load *a) /* Load the data. */ temp = tcg_temp_new_i64(); tcg_gen_addi_i64(temp, cpu_reg_sp(s, a->rn), a->imm << msz); - tcg_gen_qemu_ld_i64(temp, temp, get_mem_index(s), + clean_addr = gen_mte_check1(s, temp, false, true, msz); + + tcg_gen_qemu_ld_i64(temp, clean_addr, get_mem_index(s), s->be_data | dtype_mop[a->dtype]); /* Broadcast to *all* elements. */ From patchwork Wed Jun 3 01:13:06 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 187430 Delivered-To: patch@linaro.org Received: by 2002:a92:cf06:0:0:0:0:0 with SMTP id c6csp78214ilo; Tue, 2 Jun 2020 18:28:37 -0700 (PDT) X-Google-Smtp-Source: ABdhPJybvQwYrAfRIOhWvHbCiBS/gD16FIMqS4yvyRiywDWfQynWCJ24ko26/1SVQwtqI9XjJnYE X-Received: by 2002:a25:cbce:: with SMTP id b197mr21011660ybg.316.1591147717889; Tue, 02 Jun 2020 18:28:37 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1591147717; cv=none; d=google.com; s=arc-20160816; b=fB6IuwMwphWz1aD0hlxECGV7SwftZre/thT4VmEB0vHpWu/7CLWPc/TFKHXxQxCtt4 QsqBNDI1ApiYOcjMqAeBgvvhzflqsmnlf0J156xovYF5I0a+ahBSARyEKI+0kBoOcgrT CcVl9qpMyGeG/AmGWdlpIKfqB6M2JpakPHtEU8kO5IpsrOcdXmLkZKftYcvjMp1p/ncL 5Yoe6tPVLZ1N8t7hSo162GbGwUMcH5QnWUusnySdrozabIrvT8hxvhWCWXIJM9+BoL2z YcEPjHfWPKuLPCjpcRpRZWS6sS2sMvlTD06I09XRVVeHOq/EmvEpBWtXXxBu1spY3SZq zN1Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=2xPAi+lk3cEpSYCm7Piwq8ZTxTMvoTDJuamxmBX9DsE=; b=GUJ6q6Jf0H2DANb/ptU9bugSZNraqY9LkGqZkOQAm5pG2o48rW4VRQ4ZW/9cc2mSkr nQMhY13I9LowEiU3FRpCLU6KL2w7Z3W/ALLxRrpnATCXTGgy+VVJaq4fOAzxnn/Yl4fU dhBoPEZo+c4g1Fj++evqa/Ie6Mkbfi/uFN2vZWAqXAISc8Nxq7oBBEEprc/DCC9JwxQj JVtUrhsZMp/Baytv7/eEuMP62Z89+msD0dni3ZiNAF58rY087v+WDzLnoOKwDZ1oZAmY KDSVZ0/qzYeh3Jwd4J1f6dufjBqqvlp4vaObbcmToh9ksXLBpOckKekAFcfltjr+biY0 cmyQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=oD+lUQwn; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id o7si327977ybp.250.2020.06.02.18.28.37 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 02 Jun 2020 18:28:37 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=oD+lUQwn; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:59346 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jgICr-00078v-9U for patch@linaro.org; Tue, 02 Jun 2020 21:28:37 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:48948) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jgHyl-0004o0-QT for qemu-devel@nongnu.org; Tue, 02 Jun 2020 21:14:03 -0400 Received: from mail-pl1-x642.google.com ([2607:f8b0:4864:20::642]:42498) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1jgHyj-0003fB-Pw for qemu-devel@nongnu.org; Tue, 02 Jun 2020 21:14:03 -0400 Received: by mail-pl1-x642.google.com with SMTP id x11so194564plv.9 for ; Tue, 02 Jun 2020 18:14:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=2xPAi+lk3cEpSYCm7Piwq8ZTxTMvoTDJuamxmBX9DsE=; b=oD+lUQwnw303Lh4blXue88J2Mmhns6ykjvw7hQp5soCq7gOWNbh9348mnnqad7w+Jp CEcuO9TrRR8Das6pRDP1WkSkR6Zv84z//z73Q4tfNXJKkRFl82VAAqCG4EQgouD3mBg4 paWCnCkBoKF6wzkjxgFpd3Z7h9Xs29freaSRly5+Dqmwh8z5gbE2Q1yZ2OHWRstAU5YJ 4HGP5gSf4Y15atW7hp4yijgjCcwwUbNttxfnbzvgOZQIe61eM16yKsdgeC/i2BLLsnFj byk80A/VGfvsayIp6r/SbJk49chPRYFc16g5f41jiBorJTHuQEZLqJvEOmJPyjugI/mf U7Ug== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=2xPAi+lk3cEpSYCm7Piwq8ZTxTMvoTDJuamxmBX9DsE=; b=YQG01gk6EeFUnnTujb+1uc/1OC7Ma9ADYEvOO+nvZQS+zfrLafLA6CKlywf8X0I2+L /r0IYUk1rtyRMNA5qqKNaiA29g2gmGPKe1RLsMUNRv4Hb0RIuUUwxCheUZ4v9TYmhTDf fiZ/g4rKlw2/CEm7a3jzbD2Q29rH8ue20f8G71/RFmXYXhrMZHL81tK89HPm+RcaUyTo oiQYnZGHhCbJ3BGPBrBKzvmYX99RxRsvJXggfVe4W0/76Bh1kkdxJUAS1bV7N151rS0+ BIIC42WJdrVC7vGA5oEA9XHBAaxBRiKSmmQUlpgD9XlkDHm5Zh0lCZCny37bWD9+wnyN kcEQ== X-Gm-Message-State: AOAM530tLHa9bGOTdbf3YQoOxmQcC7KQFQqW24eErC/YNjK8WvrtPYdY UUrCmdmEdP8D8nbFd5l4JinDUA3+rR4= X-Received: by 2002:a17:902:6947:: with SMTP id k7mr27701820plt.258.1591146839586; Tue, 02 Jun 2020 18:13:59 -0700 (PDT) Received: from localhost.localdomain (174-21-143-238.tukw.qwest.net. [174.21.143.238]) by smtp.gmail.com with ESMTPSA id 3sm290067pfe.85.2020.06.02.18.13.58 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 02 Jun 2020 18:13:58 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v7 31/42] target/arm: Add mte helpers for sve scalar + int loads Date: Tue, 2 Jun 2020 18:13:06 -0700 Message-Id: <20200603011317.473934-32-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200603011317.473934-1-richard.henderson@linaro.org> References: <20200603011317.473934-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::642; envelope-from=richard.henderson@linaro.org; helo=mail-pl1-x642.google.com X-detected-operating-system: by eggs.gnu.org: No matching host in p0f cache. That's all we know. X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: peter.maydell@linaro.org, qemu-arm@nongnu.org, steplong@quicinc.com Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" Because the elements are sequential, we can eliminate many tests all at once when the tag hits TCMA, or if the page(s) are not Tagged. Signed-off-by: Richard Henderson --- target/arm/helper-sve.h | 58 ++++++++++ target/arm/internals.h | 6 + target/arm/sve_helper.c | 218 ++++++++++++++++++++++++++++++------- target/arm/translate-sve.c | 186 ++++++++++++++++++++++--------- 4 files changed, 377 insertions(+), 91 deletions(-) -- 2.25.1 diff --git a/target/arm/helper-sve.h b/target/arm/helper-sve.h index 197cd72999..d65c11c008 100644 --- a/target/arm/helper-sve.h +++ b/target/arm/helper-sve.h @@ -1199,6 +1199,64 @@ DEF_HELPER_FLAGS_4(sve_ld1sds_le_r, TCG_CALL_NO_WG, void, env, ptr, tl, i32) DEF_HELPER_FLAGS_4(sve_ld1sdu_be_r, TCG_CALL_NO_WG, void, env, ptr, tl, i32) DEF_HELPER_FLAGS_4(sve_ld1sds_be_r, TCG_CALL_NO_WG, void, env, ptr, tl, i32) +DEF_HELPER_FLAGS_4(sve_ld1bb_r_mte, TCG_CALL_NO_WG, void, env, ptr, tl, i32) +DEF_HELPER_FLAGS_4(sve_ld2bb_r_mte, TCG_CALL_NO_WG, void, env, ptr, tl, i32) +DEF_HELPER_FLAGS_4(sve_ld3bb_r_mte, TCG_CALL_NO_WG, void, env, ptr, tl, i32) +DEF_HELPER_FLAGS_4(sve_ld4bb_r_mte, TCG_CALL_NO_WG, void, env, ptr, tl, i32) + +DEF_HELPER_FLAGS_4(sve_ld1hh_le_r_mte, TCG_CALL_NO_WG, void, env, ptr, tl, i32) +DEF_HELPER_FLAGS_4(sve_ld2hh_le_r_mte, TCG_CALL_NO_WG, void, env, ptr, tl, i32) +DEF_HELPER_FLAGS_4(sve_ld3hh_le_r_mte, TCG_CALL_NO_WG, void, env, ptr, tl, i32) +DEF_HELPER_FLAGS_4(sve_ld4hh_le_r_mte, TCG_CALL_NO_WG, void, env, ptr, tl, i32) + +DEF_HELPER_FLAGS_4(sve_ld1hh_be_r_mte, TCG_CALL_NO_WG, void, env, ptr, tl, i32) +DEF_HELPER_FLAGS_4(sve_ld2hh_be_r_mte, TCG_CALL_NO_WG, void, env, ptr, tl, i32) +DEF_HELPER_FLAGS_4(sve_ld3hh_be_r_mte, TCG_CALL_NO_WG, void, env, ptr, tl, i32) +DEF_HELPER_FLAGS_4(sve_ld4hh_be_r_mte, TCG_CALL_NO_WG, void, env, ptr, tl, i32) + +DEF_HELPER_FLAGS_4(sve_ld1ss_le_r_mte, TCG_CALL_NO_WG, void, env, ptr, tl, i32) +DEF_HELPER_FLAGS_4(sve_ld2ss_le_r_mte, TCG_CALL_NO_WG, void, env, ptr, tl, i32) +DEF_HELPER_FLAGS_4(sve_ld3ss_le_r_mte, TCG_CALL_NO_WG, void, env, ptr, tl, i32) +DEF_HELPER_FLAGS_4(sve_ld4ss_le_r_mte, TCG_CALL_NO_WG, void, env, ptr, tl, i32) + +DEF_HELPER_FLAGS_4(sve_ld1ss_be_r_mte, TCG_CALL_NO_WG, void, env, ptr, tl, i32) +DEF_HELPER_FLAGS_4(sve_ld2ss_be_r_mte, TCG_CALL_NO_WG, void, env, ptr, tl, i32) +DEF_HELPER_FLAGS_4(sve_ld3ss_be_r_mte, TCG_CALL_NO_WG, void, env, ptr, tl, i32) +DEF_HELPER_FLAGS_4(sve_ld4ss_be_r_mte, TCG_CALL_NO_WG, void, env, ptr, tl, i32) + +DEF_HELPER_FLAGS_4(sve_ld1dd_le_r_mte, TCG_CALL_NO_WG, void, env, ptr, tl, i32) +DEF_HELPER_FLAGS_4(sve_ld2dd_le_r_mte, TCG_CALL_NO_WG, void, env, ptr, tl, i32) +DEF_HELPER_FLAGS_4(sve_ld3dd_le_r_mte, TCG_CALL_NO_WG, void, env, ptr, tl, i32) +DEF_HELPER_FLAGS_4(sve_ld4dd_le_r_mte, TCG_CALL_NO_WG, void, env, ptr, tl, i32) + +DEF_HELPER_FLAGS_4(sve_ld1dd_be_r_mte, TCG_CALL_NO_WG, void, env, ptr, tl, i32) +DEF_HELPER_FLAGS_4(sve_ld2dd_be_r_mte, TCG_CALL_NO_WG, void, env, ptr, tl, i32) +DEF_HELPER_FLAGS_4(sve_ld3dd_be_r_mte, TCG_CALL_NO_WG, void, env, ptr, tl, i32) +DEF_HELPER_FLAGS_4(sve_ld4dd_be_r_mte, TCG_CALL_NO_WG, void, env, ptr, tl, i32) + +DEF_HELPER_FLAGS_4(sve_ld1bhu_r_mte, TCG_CALL_NO_WG, void, env, ptr, tl, i32) +DEF_HELPER_FLAGS_4(sve_ld1bsu_r_mte, TCG_CALL_NO_WG, void, env, ptr, tl, i32) +DEF_HELPER_FLAGS_4(sve_ld1bdu_r_mte, TCG_CALL_NO_WG, void, env, ptr, tl, i32) +DEF_HELPER_FLAGS_4(sve_ld1bhs_r_mte, TCG_CALL_NO_WG, void, env, ptr, tl, i32) +DEF_HELPER_FLAGS_4(sve_ld1bss_r_mte, TCG_CALL_NO_WG, void, env, ptr, tl, i32) +DEF_HELPER_FLAGS_4(sve_ld1bds_r_mte, TCG_CALL_NO_WG, void, env, ptr, tl, i32) + +DEF_HELPER_FLAGS_4(sve_ld1hsu_le_r_mte, TCG_CALL_NO_WG, void, env, ptr, tl, i32) +DEF_HELPER_FLAGS_4(sve_ld1hdu_le_r_mte, TCG_CALL_NO_WG, void, env, ptr, tl, i32) +DEF_HELPER_FLAGS_4(sve_ld1hss_le_r_mte, TCG_CALL_NO_WG, void, env, ptr, tl, i32) +DEF_HELPER_FLAGS_4(sve_ld1hds_le_r_mte, TCG_CALL_NO_WG, void, env, ptr, tl, i32) + +DEF_HELPER_FLAGS_4(sve_ld1hsu_be_r_mte, TCG_CALL_NO_WG, void, env, ptr, tl, i32) +DEF_HELPER_FLAGS_4(sve_ld1hdu_be_r_mte, TCG_CALL_NO_WG, void, env, ptr, tl, i32) +DEF_HELPER_FLAGS_4(sve_ld1hss_be_r_mte, TCG_CALL_NO_WG, void, env, ptr, tl, i32) +DEF_HELPER_FLAGS_4(sve_ld1hds_be_r_mte, TCG_CALL_NO_WG, void, env, ptr, tl, i32) + +DEF_HELPER_FLAGS_4(sve_ld1sdu_le_r_mte, TCG_CALL_NO_WG, void, env, ptr, tl, i32) +DEF_HELPER_FLAGS_4(sve_ld1sds_le_r_mte, TCG_CALL_NO_WG, void, env, ptr, tl, i32) + +DEF_HELPER_FLAGS_4(sve_ld1sdu_be_r_mte, TCG_CALL_NO_WG, void, env, ptr, tl, i32) +DEF_HELPER_FLAGS_4(sve_ld1sds_be_r_mte, TCG_CALL_NO_WG, void, env, ptr, tl, i32) + DEF_HELPER_FLAGS_4(sve_ldff1bb_r, TCG_CALL_NO_WG, void, env, ptr, tl, i32) DEF_HELPER_FLAGS_4(sve_ldff1bhu_r, TCG_CALL_NO_WG, void, env, ptr, tl, i32) DEF_HELPER_FLAGS_4(sve_ldff1bsu_r, TCG_CALL_NO_WG, void, env, ptr, tl, i32) diff --git a/target/arm/internals.h b/target/arm/internals.h index cc485603a8..2b70446bb9 100644 --- a/target/arm/internals.h +++ b/target/arm/internals.h @@ -1310,6 +1310,12 @@ void arm_log_exception(int idx); #define LOG2_TAG_GRANULE 4 #define TAG_GRANULE (1 << LOG2_TAG_GRANULE) +/* + * The SVE simd_data field, for memory ops, contains either + * rd (5 bits) or a shift count (2 bits). + */ +#define SVE_MTEDESC_SHIFT 5 + /* Bits within a descriptor passed to the helper_mte_check* functions. */ FIELD(MTEDESC, MIDX, 0, 4) FIELD(MTEDESC, TBI, 4, 2) diff --git a/target/arm/sve_helper.c b/target/arm/sve_helper.c index 507fba37c9..ca9aacb1c0 100644 --- a/target/arm/sve_helper.c +++ b/target/arm/sve_helper.c @@ -4522,15 +4522,89 @@ static void sve_cont_ldst_watchpoints(SVEContLdSt *info, CPUARMState *env, #endif } +typedef uint64_t mte_check_fn(CPUARMState *, uint32_t, uint64_t, uintptr_t); + +static inline QEMU_ALWAYS_INLINE +void sve_cont_ldst_mte_check_int(SVEContLdSt *info, CPUARMState *env, + uint64_t *vg, target_ulong addr, int esize, + int msize, uint32_t mtedesc, uintptr_t ra, + mte_check_fn *check) +{ + intptr_t mem_off, reg_off, reg_last; + + /* Process the page only if MemAttr == Tagged. */ + if (info->page[0].attrs.target_tlb_bit1) { + mem_off = info->mem_off_first[0]; + reg_off = info->reg_off_first[0]; + reg_last = info->reg_off_split; + if (reg_last < 0) { + reg_last = info->reg_off_last[0]; + } + + do { + uint64_t pg = vg[reg_off >> 6]; + do { + if ((pg >> (reg_off & 63)) & 1) { + check(env, mtedesc, addr, ra); + } + reg_off += esize; + mem_off += msize; + } while (reg_off <= reg_last && (reg_off & 63)); + } while (reg_off <= reg_last); + } + + mem_off = info->mem_off_first[1]; + if (mem_off >= 0 && info->page[1].attrs.target_tlb_bit1) { + reg_off = info->reg_off_first[1]; + reg_last = info->reg_off_last[1]; + + do { + uint64_t pg = vg[reg_off >> 6]; + do { + if ((pg >> (reg_off & 63)) & 1) { + check(env, mtedesc, addr, ra); + } + reg_off += esize; + mem_off += msize; + } while (reg_off & 63); + } while (reg_off <= reg_last); + } +} + +typedef void sve_cont_ldst_mte_check_fn(SVEContLdSt *info, CPUARMState *env, + uint64_t *vg, target_ulong addr, + int esize, int msize, uint32_t mtedesc, + uintptr_t ra); + +static void sve_cont_ldst_mte_check1(SVEContLdSt *info, CPUARMState *env, + uint64_t *vg, target_ulong addr, + int esize, int msize, uint32_t mtedesc, + uintptr_t ra) +{ + sve_cont_ldst_mte_check_int(info, env, vg, addr, esize, msize, + mtedesc, ra, mte_check1); +} + +static void sve_cont_ldst_mte_checkN(SVEContLdSt *info, CPUARMState *env, + uint64_t *vg, target_ulong addr, + int esize, int msize, uint32_t mtedesc, + uintptr_t ra) +{ + sve_cont_ldst_mte_check_int(info, env, vg, addr, esize, msize, + mtedesc, ra, mte_checkN); +} + + /* * Common helper for all contiguous 1,2,3,4-register predicated stores. */ static inline QEMU_ALWAYS_INLINE void sve_ldN_r(CPUARMState *env, uint64_t *vg, const target_ulong addr, uint32_t desc, const uintptr_t retaddr, - const int esz, const int msz, const int N, + const int esz, const int msz, const int N, uint32_t mtedesc, sve_ldst1_host_fn *host_fn, - sve_ldst1_tlb_fn *tlb_fn) + sve_ldst1_tlb_fn *tlb_fn, + sve_cont_ldst_mte_check_fn *mte_check_fn) { const unsigned rd = simd_data(desc); const intptr_t reg_max = simd_oprsz(desc); @@ -4555,7 +4629,14 @@ void sve_ldN_r(CPUARMState *env, uint64_t *vg, const target_ulong addr, sve_cont_ldst_watchpoints(&info, env, vg, addr, 1 << esz, N << msz, BP_MEM_READ, retaddr); - /* TODO: MTE check. */ + /* + * Handle mte checks for all active elements. + * Since TBI must be set for MTE, !mtedesc => !mte_active. + */ + if (mte_check_fn && mtedesc) { + mte_check_fn(&info, env, vg, addr, 1 << esz, N << msz, + mtedesc, retaddr); + } flags = info.page[0].flags | info.page[1].flags; if (unlikely(flags != 0)) { @@ -4661,26 +4742,67 @@ void sve_ldN_r(CPUARMState *env, uint64_t *vg, const target_ulong addr, } } -#define DO_LD1_1(NAME, ESZ) \ -void HELPER(sve_##NAME##_r)(CPUARMState *env, void *vg, \ - target_ulong addr, uint32_t desc) \ -{ \ - sve_ldN_r(env, vg, addr, desc, GETPC(), ESZ, MO_8, 1, \ - sve_##NAME##_host, sve_##NAME##_tlb); \ +static inline QEMU_ALWAYS_INLINE +void sve_ldN_r_mte(CPUARMState *env, uint64_t *vg, target_ulong addr, + uint32_t desc, const uintptr_t ra, + const int esz, const int msz, const int N, + sve_ldst1_host_fn *host_fn, + sve_ldst1_tlb_fn *tlb_fn) +{ + uint32_t mtedesc = desc >> (SIMD_DATA_SHIFT + SVE_MTEDESC_SHIFT); + int bit55 = extract64(addr, 55, 1); + + /* Remove mtedesc from the normal sve descriptor. */ + desc = extract32(desc, 0, SIMD_DATA_SHIFT + SVE_MTEDESC_SHIFT); + + /* Perform gross MTE suppression early. */ + if (!tbi_check(desc, bit55) || + tcma_check(desc, bit55, allocation_tag_from_addr(addr))) { + mtedesc = 0; + } + + sve_ldN_r(env, vg, addr, desc, ra, esz, msz, N, mtedesc, host_fn, tlb_fn, + N == 1 ? sve_cont_ldst_mte_check1 : sve_cont_ldst_mte_checkN); } -#define DO_LD1_2(NAME, ESZ, MSZ) \ -void HELPER(sve_##NAME##_le_r)(CPUARMState *env, void *vg, \ - target_ulong addr, uint32_t desc) \ -{ \ - sve_ldN_r(env, vg, addr, desc, GETPC(), ESZ, MSZ, 1, \ - sve_##NAME##_le_host, sve_##NAME##_le_tlb); \ -} \ -void HELPER(sve_##NAME##_be_r)(CPUARMState *env, void *vg, \ - target_ulong addr, uint32_t desc) \ -{ \ - sve_ldN_r(env, vg, addr, desc, GETPC(), ESZ, MSZ, 1, \ - sve_##NAME##_be_host, sve_##NAME##_be_tlb); \ +#define DO_LD1_1(NAME, ESZ) \ +void HELPER(sve_##NAME##_r)(CPUARMState *env, void *vg, \ + target_ulong addr, uint32_t desc) \ +{ \ + sve_ldN_r(env, vg, addr, desc, GETPC(), ESZ, MO_8, 1, 0, \ + sve_##NAME##_host, sve_##NAME##_tlb, NULL); \ +} \ +void HELPER(sve_##NAME##_r_mte)(CPUARMState *env, void *vg, \ + target_ulong addr, uint32_t desc) \ +{ \ + sve_ldN_r_mte(env, vg, addr, desc, GETPC(), ESZ, MO_8, 1, \ + sve_##NAME##_host, sve_##NAME##_tlb); \ +} + +#define DO_LD1_2(NAME, ESZ, MSZ) \ +void HELPER(sve_##NAME##_le_r)(CPUARMState *env, void *vg, \ + target_ulong addr, uint32_t desc) \ +{ \ + sve_ldN_r(env, vg, addr, desc, GETPC(), ESZ, MSZ, 1, 0, \ + sve_##NAME##_le_host, sve_##NAME##_le_tlb, NULL); \ +} \ +void HELPER(sve_##NAME##_be_r)(CPUARMState *env, void *vg, \ + target_ulong addr, uint32_t desc) \ +{ \ + sve_ldN_r(env, vg, addr, desc, GETPC(), ESZ, MSZ, 1, 0, \ + sve_##NAME##_be_host, sve_##NAME##_be_tlb, NULL); \ +} \ +void HELPER(sve_##NAME##_le_r_mte)(CPUARMState *env, void *vg, \ + target_ulong addr, uint32_t desc) \ +{ \ + sve_ldN_r_mte(env, vg, addr, desc, GETPC(), ESZ, MSZ, 1, \ + sve_##NAME##_le_host, sve_##NAME##_le_tlb); \ +} \ +void HELPER(sve_##NAME##_be_r_mte)(CPUARMState *env, void *vg, \ + target_ulong addr, uint32_t desc) \ +{ \ + sve_ldN_r_mte(env, vg, addr, desc, GETPC(), ESZ, MSZ, 1, \ + sve_##NAME##_be_host, sve_##NAME##_be_tlb); \ } DO_LD1_1(ld1bb, MO_8) @@ -4706,26 +4828,44 @@ DO_LD1_2(ld1dd, MO_64, MO_64) #undef DO_LD1_1 #undef DO_LD1_2 -#define DO_LDN_1(N) \ -void HELPER(sve_ld##N##bb_r)(CPUARMState *env, void *vg, \ - target_ulong addr, uint32_t desc) \ -{ \ - sve_ldN_r(env, vg, addr, desc, GETPC(), MO_8, MO_8, N, \ - sve_ld1bb_host, sve_ld1bb_tlb); \ +#define DO_LDN_1(N) \ +void HELPER(sve_ld##N##bb_r)(CPUARMState *env, void *vg, \ + target_ulong addr, uint32_t desc) \ +{ \ + sve_ldN_r(env, vg, addr, desc, GETPC(), MO_8, MO_8, N, 0, \ + sve_ld1bb_host, sve_ld1bb_tlb, NULL); \ +} \ +void HELPER(sve_ld##N##bb_r_mte)(CPUARMState *env, void *vg, \ + target_ulong addr, uint32_t desc) \ +{ \ + sve_ldN_r_mte(env, vg, addr, desc, GETPC(), MO_8, MO_8, N, \ + sve_ld1bb_host, sve_ld1bb_tlb); \ } -#define DO_LDN_2(N, SUFF, ESZ) \ -void HELPER(sve_ld##N##SUFF##_le_r)(CPUARMState *env, void *vg, \ - target_ulong addr, uint32_t desc) \ -{ \ - sve_ldN_r(env, vg, addr, desc, GETPC(), ESZ, ESZ, N, \ - sve_ld1##SUFF##_le_host, sve_ld1##SUFF##_le_tlb); \ -} \ -void HELPER(sve_ld##N##SUFF##_be_r)(CPUARMState *env, void *vg, \ - target_ulong addr, uint32_t desc) \ -{ \ - sve_ldN_r(env, vg, addr, desc, GETPC(), ESZ, ESZ, N, \ - sve_ld1##SUFF##_be_host, sve_ld1##SUFF##_be_tlb); \ +#define DO_LDN_2(N, SUFF, ESZ) \ +void HELPER(sve_ld##N##SUFF##_le_r)(CPUARMState *env, void *vg, \ + target_ulong addr, uint32_t desc) \ +{ \ + sve_ldN_r(env, vg, addr, desc, GETPC(), ESZ, ESZ, N, 0, \ + sve_ld1##SUFF##_le_host, sve_ld1##SUFF##_le_tlb, NULL); \ +} \ +void HELPER(sve_ld##N##SUFF##_be_r)(CPUARMState *env, void *vg, \ + target_ulong addr, uint32_t desc) \ +{ \ + sve_ldN_r(env, vg, addr, desc, GETPC(), ESZ, ESZ, N, 0, \ + sve_ld1##SUFF##_be_host, sve_ld1##SUFF##_be_tlb, NULL); \ +} \ +void HELPER(sve_ld##N##SUFF##_le_r_mte)(CPUARMState *env, void *vg, \ + target_ulong addr, uint32_t desc) \ +{ \ + sve_ldN_r_mte(env, vg, addr, desc, GETPC(), ESZ, ESZ, N, \ + sve_ld1##SUFF##_le_host, sve_ld1##SUFF##_le_tlb); \ +} \ +void HELPER(sve_ld##N##SUFF##_be_r_mte)(CPUARMState *env, void *vg, \ + target_ulong addr, uint32_t desc) \ +{ \ + sve_ldN_r_mte(env, vg, addr, desc, GETPC(), ESZ, ESZ, N, \ + sve_ld1##SUFF##_be_host, sve_ld1##SUFF##_be_tlb); \ } DO_LDN_1(2) diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index 4b3b879815..48ef25cb32 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -4522,18 +4522,32 @@ static const uint8_t dtype_esz[16] = { }; static void do_mem_zpa(DisasContext *s, int zt, int pg, TCGv_i64 addr, - int dtype, gen_helper_gvec_mem *fn) + int dtype, uint32_t mte_n, bool is_write, + gen_helper_gvec_mem *fn) { unsigned vsz = vec_full_reg_size(s); TCGv_ptr t_pg; TCGv_i32 t_desc; - int desc; + int desc = 0; - /* For e.g. LD4, there are not enough arguments to pass all 4 + /* + * For e.g. LD4, there are not enough arguments to pass all 4 * registers as pointers, so encode the regno into the data field. * For consistency, do this even for LD1. + * TODO: mte_n check here while callers are updated. */ - desc = simd_desc(vsz, vsz, zt); + if (mte_n && s->mte_active[0]) { + int msz = dtype_msz(dtype); + + desc = FIELD_DP32(desc, MTEDESC, MIDX, get_mem_index(s)); + desc = FIELD_DP32(desc, MTEDESC, TBI, s->tbid); + desc = FIELD_DP32(desc, MTEDESC, TCMA, s->tcma); + desc = FIELD_DP32(desc, MTEDESC, WRITE, is_write); + desc = FIELD_DP32(desc, MTEDESC, ESIZE, 1 << msz); + desc = FIELD_DP32(desc, MTEDESC, TSIZE, mte_n << msz); + desc <<= SVE_MTEDESC_SHIFT; + } + desc = simd_desc(vsz, vsz, zt | desc); t_desc = tcg_const_i32(desc); t_pg = tcg_temp_new_ptr(); @@ -4547,64 +4561,132 @@ static void do_mem_zpa(DisasContext *s, int zt, int pg, TCGv_i64 addr, static void do_ld_zpa(DisasContext *s, int zt, int pg, TCGv_i64 addr, int dtype, int nreg) { - static gen_helper_gvec_mem * const fns[2][16][4] = { - /* Little-endian */ - { { gen_helper_sve_ld1bb_r, gen_helper_sve_ld2bb_r, + static gen_helper_gvec_mem * const fns[2][2][16][4] = { + { /* mte inactive, little-endian */ + { { gen_helper_sve_ld1bb_r, gen_helper_sve_ld2bb_r, gen_helper_sve_ld3bb_r, gen_helper_sve_ld4bb_r }, - { gen_helper_sve_ld1bhu_r, NULL, NULL, NULL }, - { gen_helper_sve_ld1bsu_r, NULL, NULL, NULL }, - { gen_helper_sve_ld1bdu_r, NULL, NULL, NULL }, + { gen_helper_sve_ld1bhu_r, NULL, NULL, NULL }, + { gen_helper_sve_ld1bsu_r, NULL, NULL, NULL }, + { gen_helper_sve_ld1bdu_r, NULL, NULL, NULL }, - { gen_helper_sve_ld1sds_le_r, NULL, NULL, NULL }, - { gen_helper_sve_ld1hh_le_r, gen_helper_sve_ld2hh_le_r, - gen_helper_sve_ld3hh_le_r, gen_helper_sve_ld4hh_le_r }, - { gen_helper_sve_ld1hsu_le_r, NULL, NULL, NULL }, - { gen_helper_sve_ld1hdu_le_r, NULL, NULL, NULL }, + { gen_helper_sve_ld1sds_le_r, NULL, NULL, NULL }, + { gen_helper_sve_ld1hh_le_r, gen_helper_sve_ld2hh_le_r, + gen_helper_sve_ld3hh_le_r, gen_helper_sve_ld4hh_le_r }, + { gen_helper_sve_ld1hsu_le_r, NULL, NULL, NULL }, + { gen_helper_sve_ld1hdu_le_r, NULL, NULL, NULL }, - { gen_helper_sve_ld1hds_le_r, NULL, NULL, NULL }, - { gen_helper_sve_ld1hss_le_r, NULL, NULL, NULL }, - { gen_helper_sve_ld1ss_le_r, gen_helper_sve_ld2ss_le_r, - gen_helper_sve_ld3ss_le_r, gen_helper_sve_ld4ss_le_r }, - { gen_helper_sve_ld1sdu_le_r, NULL, NULL, NULL }, + { gen_helper_sve_ld1hds_le_r, NULL, NULL, NULL }, + { gen_helper_sve_ld1hss_le_r, NULL, NULL, NULL }, + { gen_helper_sve_ld1ss_le_r, gen_helper_sve_ld2ss_le_r, + gen_helper_sve_ld3ss_le_r, gen_helper_sve_ld4ss_le_r }, + { gen_helper_sve_ld1sdu_le_r, NULL, NULL, NULL }, - { gen_helper_sve_ld1bds_r, NULL, NULL, NULL }, - { gen_helper_sve_ld1bss_r, NULL, NULL, NULL }, - { gen_helper_sve_ld1bhs_r, NULL, NULL, NULL }, - { gen_helper_sve_ld1dd_le_r, gen_helper_sve_ld2dd_le_r, - gen_helper_sve_ld3dd_le_r, gen_helper_sve_ld4dd_le_r } }, + { gen_helper_sve_ld1bds_r, NULL, NULL, NULL }, + { gen_helper_sve_ld1bss_r, NULL, NULL, NULL }, + { gen_helper_sve_ld1bhs_r, NULL, NULL, NULL }, + { gen_helper_sve_ld1dd_le_r, gen_helper_sve_ld2dd_le_r, + gen_helper_sve_ld3dd_le_r, gen_helper_sve_ld4dd_le_r } }, - /* Big-endian */ - { { gen_helper_sve_ld1bb_r, gen_helper_sve_ld2bb_r, - gen_helper_sve_ld3bb_r, gen_helper_sve_ld4bb_r }, - { gen_helper_sve_ld1bhu_r, NULL, NULL, NULL }, - { gen_helper_sve_ld1bsu_r, NULL, NULL, NULL }, - { gen_helper_sve_ld1bdu_r, NULL, NULL, NULL }, + /* mte inactive, big-endian */ + { { gen_helper_sve_ld1bb_r, gen_helper_sve_ld2bb_r, + gen_helper_sve_ld3bb_r, gen_helper_sve_ld4bb_r }, + { gen_helper_sve_ld1bhu_r, NULL, NULL, NULL }, + { gen_helper_sve_ld1bsu_r, NULL, NULL, NULL }, + { gen_helper_sve_ld1bdu_r, NULL, NULL, NULL }, - { gen_helper_sve_ld1sds_be_r, NULL, NULL, NULL }, - { gen_helper_sve_ld1hh_be_r, gen_helper_sve_ld2hh_be_r, - gen_helper_sve_ld3hh_be_r, gen_helper_sve_ld4hh_be_r }, - { gen_helper_sve_ld1hsu_be_r, NULL, NULL, NULL }, - { gen_helper_sve_ld1hdu_be_r, NULL, NULL, NULL }, + { gen_helper_sve_ld1sds_be_r, NULL, NULL, NULL }, + { gen_helper_sve_ld1hh_be_r, gen_helper_sve_ld2hh_be_r, + gen_helper_sve_ld3hh_be_r, gen_helper_sve_ld4hh_be_r }, + { gen_helper_sve_ld1hsu_be_r, NULL, NULL, NULL }, + { gen_helper_sve_ld1hdu_be_r, NULL, NULL, NULL }, - { gen_helper_sve_ld1hds_be_r, NULL, NULL, NULL }, - { gen_helper_sve_ld1hss_be_r, NULL, NULL, NULL }, - { gen_helper_sve_ld1ss_be_r, gen_helper_sve_ld2ss_be_r, - gen_helper_sve_ld3ss_be_r, gen_helper_sve_ld4ss_be_r }, - { gen_helper_sve_ld1sdu_be_r, NULL, NULL, NULL }, + { gen_helper_sve_ld1hds_be_r, NULL, NULL, NULL }, + { gen_helper_sve_ld1hss_be_r, NULL, NULL, NULL }, + { gen_helper_sve_ld1ss_be_r, gen_helper_sve_ld2ss_be_r, + gen_helper_sve_ld3ss_be_r, gen_helper_sve_ld4ss_be_r }, + { gen_helper_sve_ld1sdu_be_r, NULL, NULL, NULL }, - { gen_helper_sve_ld1bds_r, NULL, NULL, NULL }, - { gen_helper_sve_ld1bss_r, NULL, NULL, NULL }, - { gen_helper_sve_ld1bhs_r, NULL, NULL, NULL }, - { gen_helper_sve_ld1dd_be_r, gen_helper_sve_ld2dd_be_r, - gen_helper_sve_ld3dd_be_r, gen_helper_sve_ld4dd_be_r } } + { gen_helper_sve_ld1bds_r, NULL, NULL, NULL }, + { gen_helper_sve_ld1bss_r, NULL, NULL, NULL }, + { gen_helper_sve_ld1bhs_r, NULL, NULL, NULL }, + { gen_helper_sve_ld1dd_be_r, gen_helper_sve_ld2dd_be_r, + gen_helper_sve_ld3dd_be_r, gen_helper_sve_ld4dd_be_r } } }, + + { /* mte active, little-endian */ + { { gen_helper_sve_ld1bb_r_mte, + gen_helper_sve_ld2bb_r_mte, + gen_helper_sve_ld3bb_r_mte, + gen_helper_sve_ld4bb_r_mte }, + { gen_helper_sve_ld1bhu_r_mte, NULL, NULL, NULL }, + { gen_helper_sve_ld1bsu_r_mte, NULL, NULL, NULL }, + { gen_helper_sve_ld1bdu_r_mte, NULL, NULL, NULL }, + + { gen_helper_sve_ld1sds_le_r_mte, NULL, NULL, NULL }, + { gen_helper_sve_ld1hh_le_r_mte, + gen_helper_sve_ld2hh_le_r_mte, + gen_helper_sve_ld3hh_le_r_mte, + gen_helper_sve_ld4hh_le_r_mte }, + { gen_helper_sve_ld1hsu_le_r_mte, NULL, NULL, NULL }, + { gen_helper_sve_ld1hdu_le_r_mte, NULL, NULL, NULL }, + + { gen_helper_sve_ld1hds_le_r_mte, NULL, NULL, NULL }, + { gen_helper_sve_ld1hss_le_r_mte, NULL, NULL, NULL }, + { gen_helper_sve_ld1ss_le_r_mte, + gen_helper_sve_ld2ss_le_r_mte, + gen_helper_sve_ld3ss_le_r_mte, + gen_helper_sve_ld4ss_le_r_mte }, + { gen_helper_sve_ld1sdu_le_r_mte, NULL, NULL, NULL }, + + { gen_helper_sve_ld1bds_r_mte, NULL, NULL, NULL }, + { gen_helper_sve_ld1bss_r_mte, NULL, NULL, NULL }, + { gen_helper_sve_ld1bhs_r_mte, NULL, NULL, NULL }, + { gen_helper_sve_ld1dd_le_r_mte, + gen_helper_sve_ld2dd_le_r_mte, + gen_helper_sve_ld3dd_le_r_mte, + gen_helper_sve_ld4dd_le_r_mte } }, + + /* mte active, big-endian */ + { { gen_helper_sve_ld1bb_r_mte, + gen_helper_sve_ld2bb_r_mte, + gen_helper_sve_ld3bb_r_mte, + gen_helper_sve_ld4bb_r_mte }, + { gen_helper_sve_ld1bhu_r_mte, NULL, NULL, NULL }, + { gen_helper_sve_ld1bsu_r_mte, NULL, NULL, NULL }, + { gen_helper_sve_ld1bdu_r_mte, NULL, NULL, NULL }, + + { gen_helper_sve_ld1sds_be_r_mte, NULL, NULL, NULL }, + { gen_helper_sve_ld1hh_be_r_mte, + gen_helper_sve_ld2hh_be_r_mte, + gen_helper_sve_ld3hh_be_r_mte, + gen_helper_sve_ld4hh_be_r_mte }, + { gen_helper_sve_ld1hsu_be_r_mte, NULL, NULL, NULL }, + { gen_helper_sve_ld1hdu_be_r_mte, NULL, NULL, NULL }, + + { gen_helper_sve_ld1hds_be_r_mte, NULL, NULL, NULL }, + { gen_helper_sve_ld1hss_be_r_mte, NULL, NULL, NULL }, + { gen_helper_sve_ld1ss_be_r_mte, + gen_helper_sve_ld2ss_be_r_mte, + gen_helper_sve_ld3ss_be_r_mte, + gen_helper_sve_ld4ss_be_r_mte }, + { gen_helper_sve_ld1sdu_be_r_mte, NULL, NULL, NULL }, + + { gen_helper_sve_ld1bds_r_mte, NULL, NULL, NULL }, + { gen_helper_sve_ld1bss_r_mte, NULL, NULL, NULL }, + { gen_helper_sve_ld1bhs_r_mte, NULL, NULL, NULL }, + { gen_helper_sve_ld1dd_be_r_mte, + gen_helper_sve_ld2dd_be_r_mte, + gen_helper_sve_ld3dd_be_r_mte, + gen_helper_sve_ld4dd_be_r_mte } } }, }; - gen_helper_gvec_mem *fn = fns[s->be_data == MO_BE][dtype][nreg]; + gen_helper_gvec_mem *fn + = fns[s->mte_active[0]][s->be_data == MO_BE][dtype][nreg]; - /* While there are holes in the table, they are not + /* + * While there are holes in the table, they are not * accessible via the instruction encoding. */ assert(fn != NULL); - do_mem_zpa(s, zt, pg, addr, dtype, fn); + do_mem_zpa(s, zt, pg, addr, dtype, nreg, false, fn); } static bool trans_LD_zprr(DisasContext *s, arg_rprr_load *a) @@ -4686,7 +4768,7 @@ static bool trans_LDFF1_zprr(DisasContext *s, arg_rprr_load *a) TCGv_i64 addr = new_tmp_a64(s); tcg_gen_shli_i64(addr, cpu_reg(s, a->rm), dtype_msz(a->dtype)); tcg_gen_add_i64(addr, addr, cpu_reg_sp(s, a->rn)); - do_mem_zpa(s, a->rd, a->pg, addr, a->dtype, + do_mem_zpa(s, a->rd, a->pg, addr, a->dtype, 0, false, fns[s->be_data == MO_BE][a->dtype]); } return true; @@ -4745,7 +4827,7 @@ static bool trans_LDNF1_zpri(DisasContext *s, arg_rpri_load *a) TCGv_i64 addr = new_tmp_a64(s); tcg_gen_addi_i64(addr, cpu_reg_sp(s, a->rn), off); - do_mem_zpa(s, a->rd, a->pg, addr, a->dtype, + do_mem_zpa(s, a->rd, a->pg, addr, a->dtype, 0, false, fns[s->be_data == MO_BE][a->dtype]); } return true; @@ -4947,7 +5029,7 @@ static void do_st_zpa(DisasContext *s, int zt, int pg, TCGv_i64 addr, fn = fn_multiple[be][nreg - 1][msz]; } assert(fn != NULL); - do_mem_zpa(s, zt, pg, addr, msz_dtype(s, msz), fn); + do_mem_zpa(s, zt, pg, addr, msz_dtype(s, msz), 0, true, fn); } static bool trans_ST_zprr(DisasContext *s, arg_rprr_store *a) From patchwork Wed Jun 3 01:13:07 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 187438 Delivered-To: patch@linaro.org Received: by 2002:a92:cf06:0:0:0:0:0 with SMTP id c6csp80431ilo; Tue, 2 Jun 2020 18:32:01 -0700 (PDT) X-Google-Smtp-Source: ABdhPJwn5UQwN7HBXPOEamlv0r8vJA6ou7u5u8VDqE24z2dBA3++xQijK8oinWVejfbKdtgFjJ1q X-Received: by 2002:a25:d40b:: with SMTP id m11mr2711585ybf.66.1591147921390; Tue, 02 Jun 2020 18:32:01 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1591147921; cv=none; d=google.com; s=arc-20160816; b=N1BflbAisKfbHA5LrxD7lpq+3GFApr252oEC91reYP8pxsjmSg2pnpFO+wj0Dd7mQE zR7E9JIZhtosg/I5SWo/gRJtGznLfEkAqcPDJg/FYUN4Kk1IwyY5UEzuVkLPzZkO6uHd Qo6kwCEUG30VQb4lieEFLjFrrSwxls9fPX80La7/pXN/4oC+5L9+lXYUxsa5uclWv1MD rw5E+VLmXORSlN0ueYKY+kdHl8AwNmXqIE10+U/k/0w04UMFjptGEC+2OaOltdKKYeAl wejRTREDNeMi/zGXv7lHIceb6hjI/Zo/Wsk1Nzka/8+WQGK8fYbZbd3eRMmz3bOggG0z Owzg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=GwsdrF+PgP9JHzmejyjJtATyGxo4hzZ4DFhaKQL/GjQ=; b=gV20Znn6THhbRx/ueIUMsq1bWoKzFh2nEPvIug6WH4IndRjOj19Y5eHSOrwEwqLKqU ZoBSCsLE/diBBoq4Nm/Ssfb6X+XCerVU9sWzOM0YkXmCiaVxWevoscKQliYNVt/ibnEf tZUsLMcsKNwDUcmxfcFIZo07KaxaziiNxRcvGLYIszzfqIqM3gXebXs2KWJlbRCza9eF hX8YbTkfSg5v/P9BHPjiiGy0AYZbZxquJMIRUO1ahaCVPUTmBCrEZP901CECP2NPtneK TaI6qWR9pSFkfbHU/natIrJD5eL0qvrknj7yqYfb6ZZaCTV680oXaWVYgqgXTAOeBzhZ e0dg== ARC-Authentication-Results: i=1; mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=C+p5d7vO; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id b204si258028yba.489.2020.06.02.18.32.01 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 02 Jun 2020 18:32:01 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=C+p5d7vO; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:49714 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jgIG8-0006GL-RR for patch@linaro.org; Tue, 02 Jun 2020 21:32:00 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:48952) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jgHyl-0004oG-T1 for qemu-devel@nongnu.org; Tue, 02 Jun 2020 21:14:03 -0400 Received: from mail-pg1-x542.google.com ([2607:f8b0:4864:20::542]:35461) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1jgHyk-0003fM-KW for qemu-devel@nongnu.org; Tue, 02 Jun 2020 21:14:03 -0400 Received: by mail-pg1-x542.google.com with SMTP id o6so579868pgh.2 for ; Tue, 02 Jun 2020 18:14:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=GwsdrF+PgP9JHzmejyjJtATyGxo4hzZ4DFhaKQL/GjQ=; b=C+p5d7vOOzIet4P7N6NiKfbl+FMKDO2K4DvvOVkQMjyJ8R5UlUHtjPl5McxBhNpFbk HFGvcST0BsSaW6Vy2CxZE8EnyHpRFuYOJ+25AzVmQijiHWY/C1I2Aj7SEPFYXChjLvVK +7LetQSIPLoKmdY7ZQXQISXGpkcZ/9tXvAzFIvBFPjSKyhSdAp+cRZG66EFl16ea2xTp uLctT6bvbfZuX2xNJL89CjhPIKmUjZPtly83uaU6JCin/Bo2jow6TX0IoRvkylciNNdP ImTkyt0xVT6c3jTsNQj1Wj0f16ZO1hCBtQ9D9t+53lWbjGUACuP2JQqHjgMQycPA6oI8 c5dw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=GwsdrF+PgP9JHzmejyjJtATyGxo4hzZ4DFhaKQL/GjQ=; b=rKSQ/mQAb74Tiu9FfaNDKChdI31kx/smjONtPPRFpyF23T5C376iENnqGI/UG6x5pp 7LH6RGuChjsBiA3QvLA1jGPsuRMR54yKsHViOw3cclx5kCGv8qjdgtIpjhF7aPxaV4ZZ yuzpikeDX55lrmexi0xg/WzXDpQHKhI3CLtiUEdxhU26Qfl4Z1RvG56wkM0TR+pn2g5H tzC0txChQnk/qmgCWVquvWXSHjT9iadPE8FflqtVZIsODC2IhrV7gB3Ir8HjlUUnnSNA yQ005+Cgan98uIgtakmgZZM9rDBFx8CZ+3Rn8ixrB4G+Jt3muOJ1xqcq3DbP7eNI0DEc 2isA== X-Gm-Message-State: AOAM531AoArwWQrvMXzKndi3/F6PIqV/+XjOwICkub2W2s892LR8ukHs gRL9KLhASFLWhgaz4lz3m7bSfx+iPCg= X-Received: by 2002:a17:90a:a62:: with SMTP id o89mr2112612pjo.188.1591146840777; Tue, 02 Jun 2020 18:14:00 -0700 (PDT) Received: from localhost.localdomain (174-21-143-238.tukw.qwest.net. [174.21.143.238]) by smtp.gmail.com with ESMTPSA id 3sm290067pfe.85.2020.06.02.18.13.59 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 02 Jun 2020 18:14:00 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v7 32/42] target/arm: Add mte helpers for sve scalar + int stores Date: Tue, 2 Jun 2020 18:13:07 -0700 Message-Id: <20200603011317.473934-33-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200603011317.473934-1-richard.henderson@linaro.org> References: <20200603011317.473934-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::542; envelope-from=richard.henderson@linaro.org; helo=mail-pg1-x542.google.com X-detected-operating-system: by eggs.gnu.org: No matching host in p0f cache. That's all we know. X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: peter.maydell@linaro.org, qemu-arm@nongnu.org, steplong@quicinc.com Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" Because the elements are sequential, we can eliminate many tests all at once when the tag hits TCMA, or if the page(s) are not Tagged. Signed-off-by: Richard Henderson --- target/arm/helper-sve.h | 47 +++++++++++ target/arm/sve_helper.c | 95 ++++++++++++++++------ target/arm/translate-sve.c | 162 ++++++++++++++++++++++++------------- 3 files changed, 226 insertions(+), 78 deletions(-) -- 2.25.1 diff --git a/target/arm/helper-sve.h b/target/arm/helper-sve.h index d65c11c008..8cead615b1 100644 --- a/target/arm/helper-sve.h +++ b/target/arm/helper-sve.h @@ -1366,6 +1366,53 @@ DEF_HELPER_FLAGS_4(sve_st1hd_be_r, TCG_CALL_NO_WG, void, env, ptr, tl, i32) DEF_HELPER_FLAGS_4(sve_st1sd_le_r, TCG_CALL_NO_WG, void, env, ptr, tl, i32) DEF_HELPER_FLAGS_4(sve_st1sd_be_r, TCG_CALL_NO_WG, void, env, ptr, tl, i32) +DEF_HELPER_FLAGS_4(sve_st1bb_r_mte, TCG_CALL_NO_WG, void, env, ptr, tl, i32) +DEF_HELPER_FLAGS_4(sve_st2bb_r_mte, TCG_CALL_NO_WG, void, env, ptr, tl, i32) +DEF_HELPER_FLAGS_4(sve_st3bb_r_mte, TCG_CALL_NO_WG, void, env, ptr, tl, i32) +DEF_HELPER_FLAGS_4(sve_st4bb_r_mte, TCG_CALL_NO_WG, void, env, ptr, tl, i32) + +DEF_HELPER_FLAGS_4(sve_st1hh_le_r_mte, TCG_CALL_NO_WG, void, env, ptr, tl, i32) +DEF_HELPER_FLAGS_4(sve_st2hh_le_r_mte, TCG_CALL_NO_WG, void, env, ptr, tl, i32) +DEF_HELPER_FLAGS_4(sve_st3hh_le_r_mte, TCG_CALL_NO_WG, void, env, ptr, tl, i32) +DEF_HELPER_FLAGS_4(sve_st4hh_le_r_mte, TCG_CALL_NO_WG, void, env, ptr, tl, i32) + +DEF_HELPER_FLAGS_4(sve_st1hh_be_r_mte, TCG_CALL_NO_WG, void, env, ptr, tl, i32) +DEF_HELPER_FLAGS_4(sve_st2hh_be_r_mte, TCG_CALL_NO_WG, void, env, ptr, tl, i32) +DEF_HELPER_FLAGS_4(sve_st3hh_be_r_mte, TCG_CALL_NO_WG, void, env, ptr, tl, i32) +DEF_HELPER_FLAGS_4(sve_st4hh_be_r_mte, TCG_CALL_NO_WG, void, env, ptr, tl, i32) + +DEF_HELPER_FLAGS_4(sve_st1ss_le_r_mte, TCG_CALL_NO_WG, void, env, ptr, tl, i32) +DEF_HELPER_FLAGS_4(sve_st2ss_le_r_mte, TCG_CALL_NO_WG, void, env, ptr, tl, i32) +DEF_HELPER_FLAGS_4(sve_st3ss_le_r_mte, TCG_CALL_NO_WG, void, env, ptr, tl, i32) +DEF_HELPER_FLAGS_4(sve_st4ss_le_r_mte, TCG_CALL_NO_WG, void, env, ptr, tl, i32) + +DEF_HELPER_FLAGS_4(sve_st1ss_be_r_mte, TCG_CALL_NO_WG, void, env, ptr, tl, i32) +DEF_HELPER_FLAGS_4(sve_st2ss_be_r_mte, TCG_CALL_NO_WG, void, env, ptr, tl, i32) +DEF_HELPER_FLAGS_4(sve_st3ss_be_r_mte, TCG_CALL_NO_WG, void, env, ptr, tl, i32) +DEF_HELPER_FLAGS_4(sve_st4ss_be_r_mte, TCG_CALL_NO_WG, void, env, ptr, tl, i32) + +DEF_HELPER_FLAGS_4(sve_st1dd_le_r_mte, TCG_CALL_NO_WG, void, env, ptr, tl, i32) +DEF_HELPER_FLAGS_4(sve_st2dd_le_r_mte, TCG_CALL_NO_WG, void, env, ptr, tl, i32) +DEF_HELPER_FLAGS_4(sve_st3dd_le_r_mte, TCG_CALL_NO_WG, void, env, ptr, tl, i32) +DEF_HELPER_FLAGS_4(sve_st4dd_le_r_mte, TCG_CALL_NO_WG, void, env, ptr, tl, i32) + +DEF_HELPER_FLAGS_4(sve_st1dd_be_r_mte, TCG_CALL_NO_WG, void, env, ptr, tl, i32) +DEF_HELPER_FLAGS_4(sve_st2dd_be_r_mte, TCG_CALL_NO_WG, void, env, ptr, tl, i32) +DEF_HELPER_FLAGS_4(sve_st3dd_be_r_mte, TCG_CALL_NO_WG, void, env, ptr, tl, i32) +DEF_HELPER_FLAGS_4(sve_st4dd_be_r_mte, TCG_CALL_NO_WG, void, env, ptr, tl, i32) + +DEF_HELPER_FLAGS_4(sve_st1bh_r_mte, TCG_CALL_NO_WG, void, env, ptr, tl, i32) +DEF_HELPER_FLAGS_4(sve_st1bs_r_mte, TCG_CALL_NO_WG, void, env, ptr, tl, i32) +DEF_HELPER_FLAGS_4(sve_st1bd_r_mte, TCG_CALL_NO_WG, void, env, ptr, tl, i32) + +DEF_HELPER_FLAGS_4(sve_st1hs_le_r_mte, TCG_CALL_NO_WG, void, env, ptr, tl, i32) +DEF_HELPER_FLAGS_4(sve_st1hd_le_r_mte, TCG_CALL_NO_WG, void, env, ptr, tl, i32) +DEF_HELPER_FLAGS_4(sve_st1hs_be_r_mte, TCG_CALL_NO_WG, void, env, ptr, tl, i32) +DEF_HELPER_FLAGS_4(sve_st1hd_be_r_mte, TCG_CALL_NO_WG, void, env, ptr, tl, i32) + +DEF_HELPER_FLAGS_4(sve_st1sd_le_r_mte, TCG_CALL_NO_WG, void, env, ptr, tl, i32) +DEF_HELPER_FLAGS_4(sve_st1sd_be_r_mte, TCG_CALL_NO_WG, void, env, ptr, tl, i32) + DEF_HELPER_FLAGS_6(sve_ldbsu_zsu, TCG_CALL_NO_WG, void, env, ptr, ptr, ptr, tl, i32) DEF_HELPER_FLAGS_6(sve_ldhsu_le_zsu, TCG_CALL_NO_WG, diff --git a/target/arm/sve_helper.c b/target/arm/sve_helper.c index ca9aacb1c0..8d69ccd0c4 100644 --- a/target/arm/sve_helper.c +++ b/target/arm/sve_helper.c @@ -5151,11 +5151,12 @@ DO_LDFF1_LDNF1_2(dd, MO_64, MO_64) */ static inline QEMU_ALWAYS_INLINE -void sve_stN_r(CPUARMState *env, uint64_t *vg, target_ulong addr, uint32_t desc, - const uintptr_t retaddr, const int esz, - const int msz, const int N, +void sve_stN_r(CPUARMState *env, uint64_t *vg, target_ulong addr, + uint32_t desc, const uintptr_t retaddr, + const int esz, const int msz, const int N, uint32_t mtedesc, sve_ldst1_host_fn *host_fn, - sve_ldst1_tlb_fn *tlb_fn) + sve_ldst1_tlb_fn *tlb_fn, + sve_cont_ldst_mte_check_fn *mte_check_fn) { const unsigned rd = simd_data(desc); const intptr_t reg_max = simd_oprsz(desc); @@ -5177,7 +5178,14 @@ void sve_stN_r(CPUARMState *env, uint64_t *vg, target_ulong addr, uint32_t desc, sve_cont_ldst_watchpoints(&info, env, vg, addr, 1 << esz, N << msz, BP_MEM_WRITE, retaddr); - /* TODO: MTE check. */ + /* + * Handle mte checks for all active elements. + * Since TBI must be set for MTE, !mtedesc => !mte_active. + */ + if (mte_check_fn && mtedesc) { + mte_check_fn(&info, env, vg, addr, 1 << esz, N << msz, + mtedesc, retaddr); + } flags = info.page[0].flags | info.page[1].flags; if (unlikely(flags != 0)) { @@ -5271,26 +5279,67 @@ void sve_stN_r(CPUARMState *env, uint64_t *vg, target_ulong addr, uint32_t desc, } } -#define DO_STN_1(N, NAME, ESZ) \ -void HELPER(sve_st##N##NAME##_r)(CPUARMState *env, void *vg, \ - target_ulong addr, uint32_t desc) \ -{ \ - sve_stN_r(env, vg, addr, desc, GETPC(), ESZ, MO_8, N, \ - sve_st1##NAME##_host, sve_st1##NAME##_tlb); \ +static inline QEMU_ALWAYS_INLINE +void sve_stN_r_mte(CPUARMState *env, uint64_t *vg, target_ulong addr, + uint32_t desc, const uintptr_t ra, + const int esz, const int msz, const int N, + sve_ldst1_host_fn *host_fn, + sve_ldst1_tlb_fn *tlb_fn) +{ + uint32_t mtedesc = desc >> (SIMD_DATA_SHIFT + SVE_MTEDESC_SHIFT); + int bit55 = extract64(addr, 55, 1); + + /* Remove mtedesc from the normal sve descriptor. */ + desc = extract32(desc, 0, SIMD_DATA_SHIFT + SVE_MTEDESC_SHIFT); + + /* Perform gross MTE suppression early. */ + if (!tbi_check(desc, bit55) || + tcma_check(desc, bit55, allocation_tag_from_addr(addr))) { + mtedesc = 0; + } + + sve_stN_r(env, vg, addr, desc, ra, esz, msz, N, mtedesc, host_fn, tlb_fn, + N == 1 ? sve_cont_ldst_mte_check1 : sve_cont_ldst_mte_checkN); } -#define DO_STN_2(N, NAME, ESZ, MSZ) \ -void HELPER(sve_st##N##NAME##_le_r)(CPUARMState *env, void *vg, \ - target_ulong addr, uint32_t desc) \ -{ \ - sve_stN_r(env, vg, addr, desc, GETPC(), ESZ, MSZ, N, \ - sve_st1##NAME##_le_host, sve_st1##NAME##_le_tlb); \ -} \ -void HELPER(sve_st##N##NAME##_be_r)(CPUARMState *env, void *vg, \ - target_ulong addr, uint32_t desc) \ -{ \ - sve_stN_r(env, vg, addr, desc, GETPC(), ESZ, MSZ, N, \ - sve_st1##NAME##_be_host, sve_st1##NAME##_be_tlb); \ +#define DO_STN_1(N, NAME, ESZ) \ +void HELPER(sve_st##N##NAME##_r)(CPUARMState *env, void *vg, \ + target_ulong addr, uint32_t desc) \ +{ \ + sve_stN_r(env, vg, addr, desc, GETPC(), ESZ, MO_8, N, 0, \ + sve_st1##NAME##_host, sve_st1##NAME##_tlb, NULL); \ +} \ +void HELPER(sve_st##N##NAME##_r_mte)(CPUARMState *env, void *vg, \ + target_ulong addr, uint32_t desc) \ +{ \ + sve_stN_r_mte(env, vg, addr, desc, GETPC(), ESZ, MO_8, N, \ + sve_st1##NAME##_host, sve_st1##NAME##_tlb); \ +} + +#define DO_STN_2(N, NAME, ESZ, MSZ) \ +void HELPER(sve_st##N##NAME##_le_r)(CPUARMState *env, void *vg, \ + target_ulong addr, uint32_t desc) \ +{ \ + sve_stN_r(env, vg, addr, desc, GETPC(), ESZ, MSZ, N, 0, \ + sve_st1##NAME##_le_host, sve_st1##NAME##_le_tlb, NULL); \ +} \ +void HELPER(sve_st##N##NAME##_be_r)(CPUARMState *env, void *vg, \ + target_ulong addr, uint32_t desc) \ +{ \ + sve_stN_r(env, vg, addr, desc, GETPC(), ESZ, MSZ, N, 0, \ + sve_st1##NAME##_be_host, sve_st1##NAME##_be_tlb, NULL); \ +} \ +void HELPER(sve_st##N##NAME##_le_r_mte)(CPUARMState *env, void *vg, \ + target_ulong addr, uint32_t desc) \ +{ \ + sve_stN_r_mte(env, vg, addr, desc, GETPC(), ESZ, MSZ, N, \ + sve_st1##NAME##_le_host, sve_st1##NAME##_le_tlb); \ +} \ +void HELPER(sve_st##N##NAME##_be_r_mte)(CPUARMState *env, void *vg, \ + target_ulong addr, uint32_t desc) \ +{ \ + sve_stN_r_mte(env, vg, addr, desc, GETPC(), ESZ, MSZ, N, \ + sve_st1##NAME##_be_host, sve_st1##NAME##_be_tlb); \ } DO_STN_1(1, bb, MO_8) diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index 48ef25cb32..c6029b032d 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -4963,73 +4963,125 @@ static bool trans_LD1R_zpri(DisasContext *s, arg_rpri_load *a) static void do_st_zpa(DisasContext *s, int zt, int pg, TCGv_i64 addr, int msz, int esz, int nreg) { - static gen_helper_gvec_mem * const fn_single[2][4][4] = { - { { gen_helper_sve_st1bb_r, - gen_helper_sve_st1bh_r, - gen_helper_sve_st1bs_r, - gen_helper_sve_st1bd_r }, - { NULL, - gen_helper_sve_st1hh_le_r, - gen_helper_sve_st1hs_le_r, - gen_helper_sve_st1hd_le_r }, - { NULL, NULL, - gen_helper_sve_st1ss_le_r, - gen_helper_sve_st1sd_le_r }, - { NULL, NULL, NULL, - gen_helper_sve_st1dd_le_r } }, - { { gen_helper_sve_st1bb_r, - gen_helper_sve_st1bh_r, - gen_helper_sve_st1bs_r, - gen_helper_sve_st1bd_r }, - { NULL, - gen_helper_sve_st1hh_be_r, - gen_helper_sve_st1hs_be_r, - gen_helper_sve_st1hd_be_r }, - { NULL, NULL, - gen_helper_sve_st1ss_be_r, - gen_helper_sve_st1sd_be_r }, - { NULL, NULL, NULL, - gen_helper_sve_st1dd_be_r } }, + static gen_helper_gvec_mem * const fn_single[2][2][4][4] = { + { { { gen_helper_sve_st1bb_r, + gen_helper_sve_st1bh_r, + gen_helper_sve_st1bs_r, + gen_helper_sve_st1bd_r }, + { NULL, + gen_helper_sve_st1hh_le_r, + gen_helper_sve_st1hs_le_r, + gen_helper_sve_st1hd_le_r }, + { NULL, NULL, + gen_helper_sve_st1ss_le_r, + gen_helper_sve_st1sd_le_r }, + { NULL, NULL, NULL, + gen_helper_sve_st1dd_le_r } }, + { { gen_helper_sve_st1bb_r, + gen_helper_sve_st1bh_r, + gen_helper_sve_st1bs_r, + gen_helper_sve_st1bd_r }, + { NULL, + gen_helper_sve_st1hh_be_r, + gen_helper_sve_st1hs_be_r, + gen_helper_sve_st1hd_be_r }, + { NULL, NULL, + gen_helper_sve_st1ss_be_r, + gen_helper_sve_st1sd_be_r }, + { NULL, NULL, NULL, + gen_helper_sve_st1dd_be_r } } }, + + { { { gen_helper_sve_st1bb_r_mte, + gen_helper_sve_st1bh_r_mte, + gen_helper_sve_st1bs_r_mte, + gen_helper_sve_st1bd_r_mte }, + { NULL, + gen_helper_sve_st1hh_le_r_mte, + gen_helper_sve_st1hs_le_r_mte, + gen_helper_sve_st1hd_le_r_mte }, + { NULL, NULL, + gen_helper_sve_st1ss_le_r_mte, + gen_helper_sve_st1sd_le_r_mte }, + { NULL, NULL, NULL, + gen_helper_sve_st1dd_le_r_mte } }, + { { gen_helper_sve_st1bb_r_mte, + gen_helper_sve_st1bh_r_mte, + gen_helper_sve_st1bs_r_mte, + gen_helper_sve_st1bd_r_mte }, + { NULL, + gen_helper_sve_st1hh_be_r_mte, + gen_helper_sve_st1hs_be_r_mte, + gen_helper_sve_st1hd_be_r_mte }, + { NULL, NULL, + gen_helper_sve_st1ss_be_r_mte, + gen_helper_sve_st1sd_be_r_mte }, + { NULL, NULL, NULL, + gen_helper_sve_st1dd_be_r_mte } } }, }; - static gen_helper_gvec_mem * const fn_multiple[2][3][4] = { - { { gen_helper_sve_st2bb_r, - gen_helper_sve_st2hh_le_r, - gen_helper_sve_st2ss_le_r, - gen_helper_sve_st2dd_le_r }, - { gen_helper_sve_st3bb_r, - gen_helper_sve_st3hh_le_r, - gen_helper_sve_st3ss_le_r, - gen_helper_sve_st3dd_le_r }, - { gen_helper_sve_st4bb_r, - gen_helper_sve_st4hh_le_r, - gen_helper_sve_st4ss_le_r, - gen_helper_sve_st4dd_le_r } }, - { { gen_helper_sve_st2bb_r, - gen_helper_sve_st2hh_be_r, - gen_helper_sve_st2ss_be_r, - gen_helper_sve_st2dd_be_r }, - { gen_helper_sve_st3bb_r, - gen_helper_sve_st3hh_be_r, - gen_helper_sve_st3ss_be_r, - gen_helper_sve_st3dd_be_r }, - { gen_helper_sve_st4bb_r, - gen_helper_sve_st4hh_be_r, - gen_helper_sve_st4ss_be_r, - gen_helper_sve_st4dd_be_r } }, + static gen_helper_gvec_mem * const fn_multiple[2][2][3][4] = { + { { { gen_helper_sve_st2bb_r, + gen_helper_sve_st2hh_le_r, + gen_helper_sve_st2ss_le_r, + gen_helper_sve_st2dd_le_r }, + { gen_helper_sve_st3bb_r, + gen_helper_sve_st3hh_le_r, + gen_helper_sve_st3ss_le_r, + gen_helper_sve_st3dd_le_r }, + { gen_helper_sve_st4bb_r, + gen_helper_sve_st4hh_le_r, + gen_helper_sve_st4ss_le_r, + gen_helper_sve_st4dd_le_r } }, + { { gen_helper_sve_st2bb_r, + gen_helper_sve_st2hh_be_r, + gen_helper_sve_st2ss_be_r, + gen_helper_sve_st2dd_be_r }, + { gen_helper_sve_st3bb_r, + gen_helper_sve_st3hh_be_r, + gen_helper_sve_st3ss_be_r, + gen_helper_sve_st3dd_be_r }, + { gen_helper_sve_st4bb_r, + gen_helper_sve_st4hh_be_r, + gen_helper_sve_st4ss_be_r, + gen_helper_sve_st4dd_be_r } } }, + { { { gen_helper_sve_st2bb_r_mte, + gen_helper_sve_st2hh_le_r_mte, + gen_helper_sve_st2ss_le_r_mte, + gen_helper_sve_st2dd_le_r_mte }, + { gen_helper_sve_st3bb_r_mte, + gen_helper_sve_st3hh_le_r_mte, + gen_helper_sve_st3ss_le_r_mte, + gen_helper_sve_st3dd_le_r_mte }, + { gen_helper_sve_st4bb_r_mte, + gen_helper_sve_st4hh_le_r_mte, + gen_helper_sve_st4ss_le_r_mte, + gen_helper_sve_st4dd_le_r_mte } }, + { { gen_helper_sve_st2bb_r_mte, + gen_helper_sve_st2hh_be_r_mte, + gen_helper_sve_st2ss_be_r_mte, + gen_helper_sve_st2dd_be_r_mte }, + { gen_helper_sve_st3bb_r_mte, + gen_helper_sve_st3hh_be_r_mte, + gen_helper_sve_st3ss_be_r_mte, + gen_helper_sve_st3dd_be_r_mte }, + { gen_helper_sve_st4bb_r_mte, + gen_helper_sve_st4hh_be_r_mte, + gen_helper_sve_st4ss_be_r_mte, + gen_helper_sve_st4dd_be_r_mte } } }, }; gen_helper_gvec_mem *fn; int be = s->be_data == MO_BE; if (nreg == 0) { /* ST1 */ - fn = fn_single[be][msz][esz]; + fn = fn_single[s->mte_active[0]][be][msz][esz]; + nreg = 1; } else { /* ST2, ST3, ST4 -- msz == esz, enforced by encoding */ assert(msz == esz); - fn = fn_multiple[be][nreg - 1][msz]; + fn = fn_multiple[s->mte_active[0]][be][nreg - 1][msz]; } assert(fn != NULL); - do_mem_zpa(s, zt, pg, addr, msz_dtype(s, msz), 0, true, fn); + do_mem_zpa(s, zt, pg, addr, msz_dtype(s, msz), nreg, true, fn); } static bool trans_ST_zprr(DisasContext *s, arg_rprr_store *a) From patchwork Wed Jun 3 01:13:08 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 187436 Delivered-To: patch@linaro.org Received: by 2002:a92:cf06:0:0:0:0:0 with SMTP id c6csp79934ilo; Tue, 2 Jun 2020 18:31:11 -0700 (PDT) X-Google-Smtp-Source: ABdhPJw71BSFGOln7DmUnncVv7IKDxLbhImg59ZVcjCex9LOo1cTFXs+dsewxlTyjNrblhxCVsyc X-Received: by 2002:a25:9944:: with SMTP id n4mr46362145ybo.404.1591147871492; Tue, 02 Jun 2020 18:31:11 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1591147871; cv=none; d=google.com; s=arc-20160816; b=t0Ug35lYtQ/oOeRNsdFmzKR0m8sd8kebwWkOd13aiV6bIbs0Iwl1X3OX88MzV+/Aao t5JD4VyY0y4oaz5EhcDIHG9Jhhe1YxC/CnOs1mRTk3RRpXeYVyJsHQbJrJSgh+9zvv06 MAccxDn+SJsirVTNAkAxoPn+Ty3WibZMqph4LeTNYlBFPVMe/aQsV9VdWJ0QXQ0hyrtq sYhaRmM4pS6HoETBJzWwxUg99vimdGINw6zMMCZIRyeRVemfUagzvbIhKe87Z32z3ELa 1M4duHATEeZj/AseC8qdhXHkJ1LcMQgnlsf6a3gsMaXtWBrS2dlseSyJV1bBsm89oDle RhPw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=XUP7fLUyNu7aJ/azDCnH0EjQhulMay9kUPV+jtINGnw=; b=Oib8Vljq9DM1NKm0MlikIsb4AeTuteVujRo7BfwfWhvS3syj9MxnxOxJlgZdCaZpJQ iH90NevbKKdGC6LjXJjbQX5E+nz7EjOwR1Oqoiai97Vd4vmWP9uvk+Pm4rQOtJ8EXtQI OygaONrVcnTi7HHzF1iU/0PKVJ5MawDloNzXo9SBus8ytVxd7llHpRmGxbn6m5JfA4FL tLqCRK1o68cEuoYH1EayiIvjwkAhSahcm8PWVKMsxvT4DWs7gx/2ouV0gorAI7VCWrwA mtCTAfoE930tZ7Sg7h31LIWDbLIB7yjuYnAmQkJvzdAP7hQvcddxVb8RG/gtZpsKj7Pd /ckA== ARC-Authentication-Results: i=1; mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=KEhc8Qj2; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id h137si325928ybg.349.2020.06.02.18.31.11 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 02 Jun 2020 18:31:11 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=KEhc8Qj2; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:44716 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jgIFK-0004GV-ON for patch@linaro.org; Tue, 02 Jun 2020 21:31:10 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:48994) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jgHyo-0004w3-B2 for qemu-devel@nongnu.org; Tue, 02 Jun 2020 21:14:06 -0400 Received: from mail-pg1-x52b.google.com ([2607:f8b0:4864:20::52b]:36607) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1jgHym-0003fe-Ac for qemu-devel@nongnu.org; Tue, 02 Jun 2020 21:14:05 -0400 Received: by mail-pg1-x52b.google.com with SMTP id t7so572829pgt.3 for ; Tue, 02 Jun 2020 18:14:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=XUP7fLUyNu7aJ/azDCnH0EjQhulMay9kUPV+jtINGnw=; b=KEhc8Qj2H4cBL5zFIOrVv9smT58yCnDEJXTViSyMCmY9+AOVN1BsJ4weduNBFWPhG9 piPoZgYCuqFNUCpWvpxEFEslyb4xCtSJDiYydrPrpPW8ms1hfWMMeNfPdLGR/7AII8+0 WRMx0SKOtycAAfkFyq8Vt/2ueBlxcsiItj6e9jX3IVN1WLEQxDfgC65AztwWx8DE2RVB xnLEpbxgMSKt7A2N5aYPYvl8jcdB55dmzBksTee5aX+hHAcc3fKQ1zA1IOTOKdfhope2 a/e6VPCDKXHsoKG9UQrHeYKEYqnxySOoLCqq+5SEvHAPI9alD/uW+NPXM7Pm0uTvv6Qr AVcg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=XUP7fLUyNu7aJ/azDCnH0EjQhulMay9kUPV+jtINGnw=; b=QMpBkbrh1tFI5xgmjHjJ5mO46MCQIGkzLXiyYEB6F14yktHSqNr2zatfEEspoHsJmr Us5u0hMpn32Iuihgtku8mIekFBkClSHwyTITg7TWeXPoDbnYpAlhbWjzQU96w41s/fEM wbZPgK/q51bZuSkOM1rPgz8ymtb/bkqQY3+fKrvzS86XQC9mstnXqIuV0u7rLkQWe6D7 K5zvCJiJ7CabPSH2SUkIwSN65EFepTCAUrVEKbfbEVPtT1ESNKmpL53OpEwM8IJJC3d3 JHB92R1lRdALxeLdq2LZUBgmk5tDpaevYdF4cdvxnKLjawQjH8zqdLK6r3imIlyaRv+9 0iUg== X-Gm-Message-State: AOAM533cH/h1myFcHNydif9wnurXBnI3FhwaG14FykGzeh2pBEUVhntA Z8Cas/bcX/LwSBDCtFJnLmlq8GrK2gI= X-Received: by 2002:a63:5f11:: with SMTP id t17mr25479831pgb.177.1591146842122; Tue, 02 Jun 2020 18:14:02 -0700 (PDT) Received: from localhost.localdomain (174-21-143-238.tukw.qwest.net. [174.21.143.238]) by smtp.gmail.com with ESMTPSA id 3sm290067pfe.85.2020.06.02.18.14.00 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 02 Jun 2020 18:14:01 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v7 33/42] target/arm: Add mte helpers for sve scalar + int ff/nf loads Date: Tue, 2 Jun 2020 18:13:08 -0700 Message-Id: <20200603011317.473934-34-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200603011317.473934-1-richard.henderson@linaro.org> References: <20200603011317.473934-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::52b; envelope-from=richard.henderson@linaro.org; helo=mail-pg1-x52b.google.com X-detected-operating-system: by eggs.gnu.org: No matching host in p0f cache. That's all we know. X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: peter.maydell@linaro.org, qemu-arm@nongnu.org, steplong@quicinc.com Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" Because the elements are sequential, we can eliminate many tests all at once when the tag hits TCMA, or if the page(s) are not Tagged. Signed-off-by: Richard Henderson --- target/arm/helper-sve.h | 98 ++++++++++++++++ target/arm/sve_helper.c | 98 ++++++++++++++-- target/arm/translate-sve.c | 232 +++++++++++++++++++++++++------------ 3 files changed, 342 insertions(+), 86 deletions(-) -- 2.25.1 diff --git a/target/arm/helper-sve.h b/target/arm/helper-sve.h index 8cead615b1..700ed0e1cb 100644 --- a/target/arm/helper-sve.h +++ b/target/arm/helper-sve.h @@ -1288,6 +1288,55 @@ DEF_HELPER_FLAGS_4(sve_ldff1sds_be_r, TCG_CALL_NO_WG, void, env, ptr, tl, i32) DEF_HELPER_FLAGS_4(sve_ldff1dd_le_r, TCG_CALL_NO_WG, void, env, ptr, tl, i32) DEF_HELPER_FLAGS_4(sve_ldff1dd_be_r, TCG_CALL_NO_WG, void, env, ptr, tl, i32) +DEF_HELPER_FLAGS_4(sve_ldff1bb_r_mte, TCG_CALL_NO_WG, void, env, ptr, tl, i32) +DEF_HELPER_FLAGS_4(sve_ldff1bhu_r_mte, TCG_CALL_NO_WG, void, env, ptr, tl, i32) +DEF_HELPER_FLAGS_4(sve_ldff1bsu_r_mte, TCG_CALL_NO_WG, void, env, ptr, tl, i32) +DEF_HELPER_FLAGS_4(sve_ldff1bdu_r_mte, TCG_CALL_NO_WG, void, env, ptr, tl, i32) +DEF_HELPER_FLAGS_4(sve_ldff1bhs_r_mte, TCG_CALL_NO_WG, void, env, ptr, tl, i32) +DEF_HELPER_FLAGS_4(sve_ldff1bss_r_mte, TCG_CALL_NO_WG, void, env, ptr, tl, i32) +DEF_HELPER_FLAGS_4(sve_ldff1bds_r_mte, TCG_CALL_NO_WG, void, env, ptr, tl, i32) + +DEF_HELPER_FLAGS_4(sve_ldff1hh_le_r_mte, TCG_CALL_NO_WG, + void, env, ptr, tl, i32) +DEF_HELPER_FLAGS_4(sve_ldff1hsu_le_r_mte, TCG_CALL_NO_WG, + void, env, ptr, tl, i32) +DEF_HELPER_FLAGS_4(sve_ldff1hdu_le_r_mte, TCG_CALL_NO_WG, + void, env, ptr, tl, i32) +DEF_HELPER_FLAGS_4(sve_ldff1hss_le_r_mte, TCG_CALL_NO_WG, + void, env, ptr, tl, i32) +DEF_HELPER_FLAGS_4(sve_ldff1hds_le_r_mte, TCG_CALL_NO_WG, + void, env, ptr, tl, i32) + +DEF_HELPER_FLAGS_4(sve_ldff1hh_be_r_mte, TCG_CALL_NO_WG, + void, env, ptr, tl, i32) +DEF_HELPER_FLAGS_4(sve_ldff1hsu_be_r_mte, TCG_CALL_NO_WG, + void, env, ptr, tl, i32) +DEF_HELPER_FLAGS_4(sve_ldff1hdu_be_r_mte, TCG_CALL_NO_WG, + void, env, ptr, tl, i32) +DEF_HELPER_FLAGS_4(sve_ldff1hss_be_r_mte, TCG_CALL_NO_WG, + void, env, ptr, tl, i32) +DEF_HELPER_FLAGS_4(sve_ldff1hds_be_r_mte, TCG_CALL_NO_WG, + void, env, ptr, tl, i32) + +DEF_HELPER_FLAGS_4(sve_ldff1ss_le_r_mte, TCG_CALL_NO_WG, + void, env, ptr, tl, i32) +DEF_HELPER_FLAGS_4(sve_ldff1sdu_le_r_mte, TCG_CALL_NO_WG, + void, env, ptr, tl, i32) +DEF_HELPER_FLAGS_4(sve_ldff1sds_le_r_mte, TCG_CALL_NO_WG, + void, env, ptr, tl, i32) + +DEF_HELPER_FLAGS_4(sve_ldff1ss_be_r_mte, TCG_CALL_NO_WG, + void, env, ptr, tl, i32) +DEF_HELPER_FLAGS_4(sve_ldff1sdu_be_r_mte, TCG_CALL_NO_WG, + void, env, ptr, tl, i32) +DEF_HELPER_FLAGS_4(sve_ldff1sds_be_r_mte, TCG_CALL_NO_WG, + void, env, ptr, tl, i32) + +DEF_HELPER_FLAGS_4(sve_ldff1dd_le_r_mte, TCG_CALL_NO_WG, + void, env, ptr, tl, i32) +DEF_HELPER_FLAGS_4(sve_ldff1dd_be_r_mte, TCG_CALL_NO_WG, + void, env, ptr, tl, i32) + DEF_HELPER_FLAGS_4(sve_ldnf1bb_r, TCG_CALL_NO_WG, void, env, ptr, tl, i32) DEF_HELPER_FLAGS_4(sve_ldnf1bhu_r, TCG_CALL_NO_WG, void, env, ptr, tl, i32) DEF_HELPER_FLAGS_4(sve_ldnf1bsu_r, TCG_CALL_NO_WG, void, env, ptr, tl, i32) @@ -1319,6 +1368,55 @@ DEF_HELPER_FLAGS_4(sve_ldnf1sds_be_r, TCG_CALL_NO_WG, void, env, ptr, tl, i32) DEF_HELPER_FLAGS_4(sve_ldnf1dd_le_r, TCG_CALL_NO_WG, void, env, ptr, tl, i32) DEF_HELPER_FLAGS_4(sve_ldnf1dd_be_r, TCG_CALL_NO_WG, void, env, ptr, tl, i32) +DEF_HELPER_FLAGS_4(sve_ldnf1bb_r_mte, TCG_CALL_NO_WG, void, env, ptr, tl, i32) +DEF_HELPER_FLAGS_4(sve_ldnf1bhu_r_mte, TCG_CALL_NO_WG, void, env, ptr, tl, i32) +DEF_HELPER_FLAGS_4(sve_ldnf1bsu_r_mte, TCG_CALL_NO_WG, void, env, ptr, tl, i32) +DEF_HELPER_FLAGS_4(sve_ldnf1bdu_r_mte, TCG_CALL_NO_WG, void, env, ptr, tl, i32) +DEF_HELPER_FLAGS_4(sve_ldnf1bhs_r_mte, TCG_CALL_NO_WG, void, env, ptr, tl, i32) +DEF_HELPER_FLAGS_4(sve_ldnf1bss_r_mte, TCG_CALL_NO_WG, void, env, ptr, tl, i32) +DEF_HELPER_FLAGS_4(sve_ldnf1bds_r_mte, TCG_CALL_NO_WG, void, env, ptr, tl, i32) + +DEF_HELPER_FLAGS_4(sve_ldnf1hh_le_r_mte, TCG_CALL_NO_WG, + void, env, ptr, tl, i32) +DEF_HELPER_FLAGS_4(sve_ldnf1hsu_le_r_mte, TCG_CALL_NO_WG, + void, env, ptr, tl, i32) +DEF_HELPER_FLAGS_4(sve_ldnf1hdu_le_r_mte, TCG_CALL_NO_WG, + void, env, ptr, tl, i32) +DEF_HELPER_FLAGS_4(sve_ldnf1hss_le_r_mte, TCG_CALL_NO_WG, + void, env, ptr, tl, i32) +DEF_HELPER_FLAGS_4(sve_ldnf1hds_le_r_mte, TCG_CALL_NO_WG, + void, env, ptr, tl, i32) + +DEF_HELPER_FLAGS_4(sve_ldnf1hh_be_r_mte, TCG_CALL_NO_WG, + void, env, ptr, tl, i32) +DEF_HELPER_FLAGS_4(sve_ldnf1hsu_be_r_mte, TCG_CALL_NO_WG, + void, env, ptr, tl, i32) +DEF_HELPER_FLAGS_4(sve_ldnf1hdu_be_r_mte, TCG_CALL_NO_WG, + void, env, ptr, tl, i32) +DEF_HELPER_FLAGS_4(sve_ldnf1hss_be_r_mte, TCG_CALL_NO_WG, + void, env, ptr, tl, i32) +DEF_HELPER_FLAGS_4(sve_ldnf1hds_be_r_mte, TCG_CALL_NO_WG, + void, env, ptr, tl, i32) + +DEF_HELPER_FLAGS_4(sve_ldnf1ss_le_r_mte, TCG_CALL_NO_WG, + void, env, ptr, tl, i32) +DEF_HELPER_FLAGS_4(sve_ldnf1sdu_le_r_mte, TCG_CALL_NO_WG, + void, env, ptr, tl, i32) +DEF_HELPER_FLAGS_4(sve_ldnf1sds_le_r_mte, TCG_CALL_NO_WG, + void, env, ptr, tl, i32) + +DEF_HELPER_FLAGS_4(sve_ldnf1ss_be_r_mte, TCG_CALL_NO_WG, + void, env, ptr, tl, i32) +DEF_HELPER_FLAGS_4(sve_ldnf1sdu_be_r_mte, TCG_CALL_NO_WG, + void, env, ptr, tl, i32) +DEF_HELPER_FLAGS_4(sve_ldnf1sds_be_r_mte, TCG_CALL_NO_WG, + void, env, ptr, tl, i32) + +DEF_HELPER_FLAGS_4(sve_ldnf1dd_le_r_mte, TCG_CALL_NO_WG, + void, env, ptr, tl, i32) +DEF_HELPER_FLAGS_4(sve_ldnf1dd_be_r_mte, TCG_CALL_NO_WG, + void, env, ptr, tl, i32) + DEF_HELPER_FLAGS_4(sve_st1bb_r, TCG_CALL_NO_WG, void, env, ptr, tl, i32) DEF_HELPER_FLAGS_4(sve_st2bb_r, TCG_CALL_NO_WG, void, env, ptr, tl, i32) DEF_HELPER_FLAGS_4(sve_st3bb_r, TCG_CALL_NO_WG, void, env, ptr, tl, i32) diff --git a/target/arm/sve_helper.c b/target/arm/sve_helper.c index 8d69ccd0c4..85bf8e5de3 100644 --- a/target/arm/sve_helper.c +++ b/target/arm/sve_helper.c @@ -4923,7 +4923,7 @@ static void record_fault(CPUARMState *env, uintptr_t i, uintptr_t oprsz) */ static inline QEMU_ALWAYS_INLINE void sve_ldnfff1_r(CPUARMState *env, void *vg, const target_ulong addr, - uint32_t desc, const uintptr_t retaddr, + uint32_t desc, const uintptr_t retaddr, uint32_t mtedesc, const int esz, const int msz, const SVEContFault fault, sve_ldst1_host_fn *host_fn, sve_ldst1_tlb_fn *tlb_fn) @@ -4955,13 +4955,22 @@ void sve_ldnfff1_r(CPUARMState *env, void *vg, const target_ulong addr, mem_off = info.mem_off_first[0]; flags = info.page[0].flags; + /* Since TBI must be set for MTE, !mtedesc => !mte_active. */ + if (info.page[0].attrs.target_tlb_bit1) { + mtedesc = 0; + } + if (fault == FAULT_FIRST) { + /* Trapping mte check for the first-fault element. */ + if (mtedesc) { + mte_check1(env, mtedesc, addr + mem_off, retaddr); + } + /* * Special handling of the first active element, * if it crosses a page boundary or is MMIO. */ bool is_split = mem_off == info.mem_off_split; - /* TODO: MTE check. */ if (unlikely(flags != 0) || unlikely(is_split)) { /* * Use the slow path for cross-page handling. @@ -4997,7 +5006,10 @@ void sve_ldnfff1_r(CPUARMState *env, void *vg, const target_ulong addr, /* Watchpoint hit, see below. */ goto do_fault; } - /* TODO: MTE check. */ + if (mtedesc && + !mte_probe1(env, mtedesc, addr + mem_off, retaddr)) { + goto do_fault; + } /* * Use the slow path for cross-page handling. * This is RAM, without a watchpoint, and will not trap. @@ -5045,7 +5057,10 @@ void sve_ldnfff1_r(CPUARMState *env, void *vg, const target_ulong addr, & BP_MEM_READ)) { goto do_fault; } - /* TODO: MTE check. */ + if (mtedesc && + !mte_probe1(env, mtedesc, addr + mem_off, retaddr)) { + goto do_fault; + } host_fn(vd, reg_off, host + mem_off); } reg_off += 1 << esz; @@ -5083,44 +5098,103 @@ void sve_ldnfff1_r(CPUARMState *env, void *vg, const target_ulong addr, record_fault(env, reg_off, reg_max); } -#define DO_LDFF1_LDNF1_1(PART, ESZ) \ +static inline QEMU_ALWAYS_INLINE +void sve_ldnfff1_r_mte(CPUARMState *env, void *vg, target_ulong addr, + uint32_t desc, const uintptr_t retaddr, + const int esz, const int msz, const SVEContFault fault, + sve_ldst1_host_fn *host_fn, + sve_ldst1_tlb_fn *tlb_fn) +{ + uint32_t mtedesc = desc >> (SIMD_DATA_SHIFT + SVE_MTEDESC_SHIFT); + int bit55 = extract64(addr, 55, 1); + + /* Remove mtedesc from the normal sve descriptor. */ + desc = extract32(desc, 0, SIMD_DATA_SHIFT + SVE_MTEDESC_SHIFT); + + /* Perform gross MTE suppression early. */ + if (!tbi_check(desc, bit55) || + tcma_check(desc, bit55, allocation_tag_from_addr(addr))) { + mtedesc = 0; + } + + sve_ldnfff1_r(env, vg, addr, desc, retaddr, mtedesc, + esz, msz, fault, host_fn, tlb_fn); +} + +#define DO_LDFF1_LDNF1_1(PART, ESZ) \ void HELPER(sve_ldff1##PART##_r)(CPUARMState *env, void *vg, \ target_ulong addr, uint32_t desc) \ { \ - sve_ldnfff1_r(env, vg, addr, desc, GETPC(), ESZ, MO_8, FAULT_FIRST, \ + sve_ldnfff1_r(env, vg, addr, desc, GETPC(), 0, ESZ, MO_8, FAULT_FIRST, \ sve_ld1##PART##_host, sve_ld1##PART##_tlb); \ } \ void HELPER(sve_ldnf1##PART##_r)(CPUARMState *env, void *vg, \ target_ulong addr, uint32_t desc) \ { \ - sve_ldnfff1_r(env, vg, addr, desc, GETPC(), ESZ, MO_8, FAULT_NO, \ + sve_ldnfff1_r(env, vg, addr, desc, GETPC(), 0, ESZ, MO_8, FAULT_NO, \ + sve_ld1##PART##_host, sve_ld1##PART##_tlb); \ +} \ +void HELPER(sve_ldff1##PART##_r_mte)(CPUARMState *env, void *vg, \ + target_ulong addr, uint32_t desc) \ +{ \ + sve_ldnfff1_r_mte(env, vg, addr, desc, GETPC(), ESZ, MO_8, FAULT_FIRST, \ + sve_ld1##PART##_host, sve_ld1##PART##_tlb); \ +} \ +void HELPER(sve_ldnf1##PART##_r_mte)(CPUARMState *env, void *vg, \ + target_ulong addr, uint32_t desc) \ +{ \ + sve_ldnfff1_r_mte(env, vg, addr, desc, GETPC(), ESZ, MO_8, FAULT_NO, \ sve_ld1##PART##_host, sve_ld1##PART##_tlb); \ } -#define DO_LDFF1_LDNF1_2(PART, ESZ, MSZ) \ +#define DO_LDFF1_LDNF1_2(PART, ESZ, MSZ) \ void HELPER(sve_ldff1##PART##_le_r)(CPUARMState *env, void *vg, \ target_ulong addr, uint32_t desc) \ { \ - sve_ldnfff1_r(env, vg, addr, desc, GETPC(), ESZ, MSZ, FAULT_FIRST, \ + sve_ldnfff1_r(env, vg, addr, desc, GETPC(), 0, ESZ, MSZ, FAULT_FIRST, \ sve_ld1##PART##_le_host, sve_ld1##PART##_le_tlb); \ } \ void HELPER(sve_ldnf1##PART##_le_r)(CPUARMState *env, void *vg, \ target_ulong addr, uint32_t desc) \ { \ - sve_ldnfff1_r(env, vg, addr, desc, GETPC(), ESZ, MSZ, FAULT_NO, \ + sve_ldnfff1_r(env, vg, addr, desc, GETPC(), 0, ESZ, MSZ, FAULT_NO, \ sve_ld1##PART##_le_host, sve_ld1##PART##_le_tlb); \ } \ void HELPER(sve_ldff1##PART##_be_r)(CPUARMState *env, void *vg, \ target_ulong addr, uint32_t desc) \ { \ - sve_ldnfff1_r(env, vg, addr, desc, GETPC(), ESZ, MSZ, FAULT_FIRST, \ + sve_ldnfff1_r(env, vg, addr, desc, GETPC(), 0, ESZ, MSZ, FAULT_FIRST, \ sve_ld1##PART##_be_host, sve_ld1##PART##_be_tlb); \ } \ void HELPER(sve_ldnf1##PART##_be_r)(CPUARMState *env, void *vg, \ target_ulong addr, uint32_t desc) \ { \ - sve_ldnfff1_r(env, vg, addr, desc, GETPC(), ESZ, MSZ, FAULT_NO, \ + sve_ldnfff1_r(env, vg, addr, desc, GETPC(), 0, ESZ, MSZ, FAULT_NO, \ sve_ld1##PART##_be_host, sve_ld1##PART##_be_tlb); \ +} \ +void HELPER(sve_ldff1##PART##_le_r_mte)(CPUARMState *env, void *vg, \ + target_ulong addr, uint32_t desc) \ +{ \ + sve_ldnfff1_r_mte(env, vg, addr, desc, GETPC(), ESZ, MSZ, FAULT_FIRST, \ + sve_ld1##PART##_le_host, sve_ld1##PART##_le_tlb); \ +} \ +void HELPER(sve_ldnf1##PART##_le_r_mte)(CPUARMState *env, void *vg, \ + target_ulong addr, uint32_t desc) \ +{ \ + sve_ldnfff1_r_mte(env, vg, addr, desc, GETPC(), ESZ, MSZ, FAULT_NO, \ + sve_ld1##PART##_le_host, sve_ld1##PART##_le_tlb); \ +} \ +void HELPER(sve_ldff1##PART##_be_r_mte)(CPUARMState *env, void *vg, \ + target_ulong addr, uint32_t desc) \ +{ \ + sve_ldnfff1_r_mte(env, vg, addr, desc, GETPC(), ESZ, MSZ, FAULT_FIRST, \ + sve_ld1##PART##_be_host, sve_ld1##PART##_be_tlb); \ +} \ +void HELPER(sve_ldnf1##PART##_be_r_mte)(CPUARMState *env, void *vg, \ + target_ulong addr, uint32_t desc) \ +{ \ + sve_ldnfff1_r_mte(env, vg, addr, desc, GETPC(), ESZ, MSZ, FAULT_NO, \ + sve_ld1##PART##_be_host, sve_ld1##PART##_be_tlb); \ } DO_LDFF1_LDNF1_1(bb, MO_8) diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index c6029b032d..a1dc325104 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -4720,104 +4720,188 @@ static bool trans_LD_zpri(DisasContext *s, arg_rpri_load *a) static bool trans_LDFF1_zprr(DisasContext *s, arg_rprr_load *a) { - static gen_helper_gvec_mem * const fns[2][16] = { - /* Little-endian */ - { gen_helper_sve_ldff1bb_r, - gen_helper_sve_ldff1bhu_r, - gen_helper_sve_ldff1bsu_r, - gen_helper_sve_ldff1bdu_r, + static gen_helper_gvec_mem * const fns[2][2][16] = { + { /* mte inactive, little-endian */ + { gen_helper_sve_ldff1bb_r, + gen_helper_sve_ldff1bhu_r, + gen_helper_sve_ldff1bsu_r, + gen_helper_sve_ldff1bdu_r, - gen_helper_sve_ldff1sds_le_r, - gen_helper_sve_ldff1hh_le_r, - gen_helper_sve_ldff1hsu_le_r, - gen_helper_sve_ldff1hdu_le_r, + gen_helper_sve_ldff1sds_le_r, + gen_helper_sve_ldff1hh_le_r, + gen_helper_sve_ldff1hsu_le_r, + gen_helper_sve_ldff1hdu_le_r, - gen_helper_sve_ldff1hds_le_r, - gen_helper_sve_ldff1hss_le_r, - gen_helper_sve_ldff1ss_le_r, - gen_helper_sve_ldff1sdu_le_r, + gen_helper_sve_ldff1hds_le_r, + gen_helper_sve_ldff1hss_le_r, + gen_helper_sve_ldff1ss_le_r, + gen_helper_sve_ldff1sdu_le_r, - gen_helper_sve_ldff1bds_r, - gen_helper_sve_ldff1bss_r, - gen_helper_sve_ldff1bhs_r, - gen_helper_sve_ldff1dd_le_r }, + gen_helper_sve_ldff1bds_r, + gen_helper_sve_ldff1bss_r, + gen_helper_sve_ldff1bhs_r, + gen_helper_sve_ldff1dd_le_r }, - /* Big-endian */ - { gen_helper_sve_ldff1bb_r, - gen_helper_sve_ldff1bhu_r, - gen_helper_sve_ldff1bsu_r, - gen_helper_sve_ldff1bdu_r, + /* mte inactive, big-endian */ + { gen_helper_sve_ldff1bb_r, + gen_helper_sve_ldff1bhu_r, + gen_helper_sve_ldff1bsu_r, + gen_helper_sve_ldff1bdu_r, - gen_helper_sve_ldff1sds_be_r, - gen_helper_sve_ldff1hh_be_r, - gen_helper_sve_ldff1hsu_be_r, - gen_helper_sve_ldff1hdu_be_r, + gen_helper_sve_ldff1sds_be_r, + gen_helper_sve_ldff1hh_be_r, + gen_helper_sve_ldff1hsu_be_r, + gen_helper_sve_ldff1hdu_be_r, - gen_helper_sve_ldff1hds_be_r, - gen_helper_sve_ldff1hss_be_r, - gen_helper_sve_ldff1ss_be_r, - gen_helper_sve_ldff1sdu_be_r, + gen_helper_sve_ldff1hds_be_r, + gen_helper_sve_ldff1hss_be_r, + gen_helper_sve_ldff1ss_be_r, + gen_helper_sve_ldff1sdu_be_r, - gen_helper_sve_ldff1bds_r, - gen_helper_sve_ldff1bss_r, - gen_helper_sve_ldff1bhs_r, - gen_helper_sve_ldff1dd_be_r }, + gen_helper_sve_ldff1bds_r, + gen_helper_sve_ldff1bss_r, + gen_helper_sve_ldff1bhs_r, + gen_helper_sve_ldff1dd_be_r } }, + + { /* mte active, little-endian */ + { gen_helper_sve_ldff1bb_r_mte, + gen_helper_sve_ldff1bhu_r_mte, + gen_helper_sve_ldff1bsu_r_mte, + gen_helper_sve_ldff1bdu_r_mte, + + gen_helper_sve_ldff1sds_le_r_mte, + gen_helper_sve_ldff1hh_le_r_mte, + gen_helper_sve_ldff1hsu_le_r_mte, + gen_helper_sve_ldff1hdu_le_r_mte, + + gen_helper_sve_ldff1hds_le_r_mte, + gen_helper_sve_ldff1hss_le_r_mte, + gen_helper_sve_ldff1ss_le_r_mte, + gen_helper_sve_ldff1sdu_le_r_mte, + + gen_helper_sve_ldff1bds_r_mte, + gen_helper_sve_ldff1bss_r_mte, + gen_helper_sve_ldff1bhs_r_mte, + gen_helper_sve_ldff1dd_le_r_mte }, + + /* mte active, big-endian */ + { gen_helper_sve_ldff1bb_r_mte, + gen_helper_sve_ldff1bhu_r_mte, + gen_helper_sve_ldff1bsu_r_mte, + gen_helper_sve_ldff1bdu_r_mte, + + gen_helper_sve_ldff1sds_be_r_mte, + gen_helper_sve_ldff1hh_be_r_mte, + gen_helper_sve_ldff1hsu_be_r_mte, + gen_helper_sve_ldff1hdu_be_r_mte, + + gen_helper_sve_ldff1hds_be_r_mte, + gen_helper_sve_ldff1hss_be_r_mte, + gen_helper_sve_ldff1ss_be_r_mte, + gen_helper_sve_ldff1sdu_be_r_mte, + + gen_helper_sve_ldff1bds_r_mte, + gen_helper_sve_ldff1bss_r_mte, + gen_helper_sve_ldff1bhs_r_mte, + gen_helper_sve_ldff1dd_be_r_mte } }, }; if (sve_access_check(s)) { TCGv_i64 addr = new_tmp_a64(s); tcg_gen_shli_i64(addr, cpu_reg(s, a->rm), dtype_msz(a->dtype)); tcg_gen_add_i64(addr, addr, cpu_reg_sp(s, a->rn)); - do_mem_zpa(s, a->rd, a->pg, addr, a->dtype, 0, false, - fns[s->be_data == MO_BE][a->dtype]); + do_mem_zpa(s, a->rd, a->pg, addr, a->dtype, 1, false, + fns[s->mte_active[0]][s->be_data == MO_BE][a->dtype]); } return true; } static bool trans_LDNF1_zpri(DisasContext *s, arg_rpri_load *a) { - static gen_helper_gvec_mem * const fns[2][16] = { - /* Little-endian */ - { gen_helper_sve_ldnf1bb_r, - gen_helper_sve_ldnf1bhu_r, - gen_helper_sve_ldnf1bsu_r, - gen_helper_sve_ldnf1bdu_r, + static gen_helper_gvec_mem * const fns[2][2][16] = { + { /* mte inactive, little-endian */ + { gen_helper_sve_ldnf1bb_r, + gen_helper_sve_ldnf1bhu_r, + gen_helper_sve_ldnf1bsu_r, + gen_helper_sve_ldnf1bdu_r, - gen_helper_sve_ldnf1sds_le_r, - gen_helper_sve_ldnf1hh_le_r, - gen_helper_sve_ldnf1hsu_le_r, - gen_helper_sve_ldnf1hdu_le_r, + gen_helper_sve_ldnf1sds_le_r, + gen_helper_sve_ldnf1hh_le_r, + gen_helper_sve_ldnf1hsu_le_r, + gen_helper_sve_ldnf1hdu_le_r, - gen_helper_sve_ldnf1hds_le_r, - gen_helper_sve_ldnf1hss_le_r, - gen_helper_sve_ldnf1ss_le_r, - gen_helper_sve_ldnf1sdu_le_r, + gen_helper_sve_ldnf1hds_le_r, + gen_helper_sve_ldnf1hss_le_r, + gen_helper_sve_ldnf1ss_le_r, + gen_helper_sve_ldnf1sdu_le_r, - gen_helper_sve_ldnf1bds_r, - gen_helper_sve_ldnf1bss_r, - gen_helper_sve_ldnf1bhs_r, - gen_helper_sve_ldnf1dd_le_r }, + gen_helper_sve_ldnf1bds_r, + gen_helper_sve_ldnf1bss_r, + gen_helper_sve_ldnf1bhs_r, + gen_helper_sve_ldnf1dd_le_r }, - /* Big-endian */ - { gen_helper_sve_ldnf1bb_r, - gen_helper_sve_ldnf1bhu_r, - gen_helper_sve_ldnf1bsu_r, - gen_helper_sve_ldnf1bdu_r, + /* mte inactive, big-endian */ + { gen_helper_sve_ldnf1bb_r, + gen_helper_sve_ldnf1bhu_r, + gen_helper_sve_ldnf1bsu_r, + gen_helper_sve_ldnf1bdu_r, - gen_helper_sve_ldnf1sds_be_r, - gen_helper_sve_ldnf1hh_be_r, - gen_helper_sve_ldnf1hsu_be_r, - gen_helper_sve_ldnf1hdu_be_r, + gen_helper_sve_ldnf1sds_be_r, + gen_helper_sve_ldnf1hh_be_r, + gen_helper_sve_ldnf1hsu_be_r, + gen_helper_sve_ldnf1hdu_be_r, - gen_helper_sve_ldnf1hds_be_r, - gen_helper_sve_ldnf1hss_be_r, - gen_helper_sve_ldnf1ss_be_r, - gen_helper_sve_ldnf1sdu_be_r, + gen_helper_sve_ldnf1hds_be_r, + gen_helper_sve_ldnf1hss_be_r, + gen_helper_sve_ldnf1ss_be_r, + gen_helper_sve_ldnf1sdu_be_r, - gen_helper_sve_ldnf1bds_r, - gen_helper_sve_ldnf1bss_r, - gen_helper_sve_ldnf1bhs_r, - gen_helper_sve_ldnf1dd_be_r }, + gen_helper_sve_ldnf1bds_r, + gen_helper_sve_ldnf1bss_r, + gen_helper_sve_ldnf1bhs_r, + gen_helper_sve_ldnf1dd_be_r } }, + + { /* mte inactive, little-endian */ + { gen_helper_sve_ldnf1bb_r_mte, + gen_helper_sve_ldnf1bhu_r_mte, + gen_helper_sve_ldnf1bsu_r_mte, + gen_helper_sve_ldnf1bdu_r_mte, + + gen_helper_sve_ldnf1sds_le_r_mte, + gen_helper_sve_ldnf1hh_le_r_mte, + gen_helper_sve_ldnf1hsu_le_r_mte, + gen_helper_sve_ldnf1hdu_le_r_mte, + + gen_helper_sve_ldnf1hds_le_r_mte, + gen_helper_sve_ldnf1hss_le_r_mte, + gen_helper_sve_ldnf1ss_le_r_mte, + gen_helper_sve_ldnf1sdu_le_r_mte, + + gen_helper_sve_ldnf1bds_r_mte, + gen_helper_sve_ldnf1bss_r_mte, + gen_helper_sve_ldnf1bhs_r_mte, + gen_helper_sve_ldnf1dd_le_r_mte }, + + /* mte inactive, big-endian */ + { gen_helper_sve_ldnf1bb_r_mte, + gen_helper_sve_ldnf1bhu_r_mte, + gen_helper_sve_ldnf1bsu_r_mte, + gen_helper_sve_ldnf1bdu_r_mte, + + gen_helper_sve_ldnf1sds_be_r_mte, + gen_helper_sve_ldnf1hh_be_r_mte, + gen_helper_sve_ldnf1hsu_be_r_mte, + gen_helper_sve_ldnf1hdu_be_r_mte, + + gen_helper_sve_ldnf1hds_be_r_mte, + gen_helper_sve_ldnf1hss_be_r_mte, + gen_helper_sve_ldnf1ss_be_r_mte, + gen_helper_sve_ldnf1sdu_be_r_mte, + + gen_helper_sve_ldnf1bds_r_mte, + gen_helper_sve_ldnf1bss_r_mte, + gen_helper_sve_ldnf1bhs_r_mte, + gen_helper_sve_ldnf1dd_be_r_mte } }, }; if (sve_access_check(s)) { @@ -4827,8 +4911,8 @@ static bool trans_LDNF1_zpri(DisasContext *s, arg_rpri_load *a) TCGv_i64 addr = new_tmp_a64(s); tcg_gen_addi_i64(addr, cpu_reg_sp(s, a->rn), off); - do_mem_zpa(s, a->rd, a->pg, addr, a->dtype, 0, false, - fns[s->be_data == MO_BE][a->dtype]); + do_mem_zpa(s, a->rd, a->pg, addr, a->dtype, 1, false, + fns[s->mte_active[0]][s->be_data == MO_BE][a->dtype]); } return true; } From patchwork Wed Jun 3 01:13:09 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 187432 Delivered-To: patch@linaro.org Received: by 2002:a92:cf06:0:0:0:0:0 with SMTP id c6csp78285ilo; Tue, 2 Jun 2020 18:28:44 -0700 (PDT) X-Google-Smtp-Source: ABdhPJy9hNOAbV0lEHIy7VTsnHan7LtBOZ+UM2ygekej71P4K+ns6cxlm2q6Vk2lVPf5dvKc89Bf X-Received: by 2002:a25:6a54:: with SMTP id f81mr41570496ybc.479.1591147724356; Tue, 02 Jun 2020 18:28:44 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1591147724; cv=none; d=google.com; s=arc-20160816; b=lKctwv9B18FuRUdNGirl85qL3K6CSXZ9WBIVQ1cR9q536KeuhSXJau2heYMNrwRTfd YNrQVbP+4YNL7k4gdLEfbwDX2eR+PjjeOTwrwPLqr6s5LLX/JiQqZtezcY+KVVICIQVL ZBdKVWimxdhH+AczLCwk9nRwK6SstF20cWaQdAOV3qIHOtk+PHHqJKY2YWgazP2tgqV3 hmc4gfwDFNZoVH83FXA/ZDZTuDTXbuUnbe8BAjiJhi748+ETF41IPLDdENBWZK6HtlZh xbP+7gxbhQmSQUdXjUweADaxfQS59+mUX7PvBp0Bl+11PCMSjnygVwZ8fgRgk7kpJ7sI udxg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=Dm0P0EGuKeATfeCtNBK49sionbdZGtbXaMOAetX5rZs=; b=fWN+kB2IQ5c9i6OXrczflsNaohJx6qdhXexPhPlIs3Amjmiq6sMynVG4DCpD4sGoeo N8leHivGqPfQCr6XZyHaNXNmVec2yzFiM5ra1LC9D2vjV5dhBni4kIve+SlBrLq7qeuD 2uKP6KkdTa65TXhGs0KBkDgD5PSlxPyIqb3dEkQiw29E3qeJ7tOA3J9ip5l9IKkTisHF sWXcq5d/j/ASXtBYyGWYVjKjL3mqBo5EX9gGYvmI6mjb9pDqR17b4llF4ZCzlSlum0mf f8/zJIOrQ8xgiEmiXM92PJkncwWkxxHONEYyOGaNAe4q/IUs1RImoDrVtpX0Y7C2fWTu K0uA== ARC-Authentication-Results: i=1; mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=bQqRAbWM; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id 73si360578ybf.30.2020.06.02.18.28.44 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 02 Jun 2020 18:28:44 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=bQqRAbWM; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:57474 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jgICx-0006OW-IM for patch@linaro.org; Tue, 02 Jun 2020 21:28:43 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:48992) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jgHyo-0004vP-7c for qemu-devel@nongnu.org; Tue, 02 Jun 2020 21:14:06 -0400 Received: from mail-pg1-x541.google.com ([2607:f8b0:4864:20::541]:35918) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1jgHyn-0003fw-AW for qemu-devel@nongnu.org; Tue, 02 Jun 2020 21:14:05 -0400 Received: by mail-pg1-x541.google.com with SMTP id t7so572871pgt.3 for ; Tue, 02 Jun 2020 18:14:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=Dm0P0EGuKeATfeCtNBK49sionbdZGtbXaMOAetX5rZs=; b=bQqRAbWMVYn9fQvJ5IJIkMjuuJ0sEMoCMhU679pbtyLpBMtbafrB5d/aW0BJRyzqn5 hStVgnyCxQYrnwNY1ZZ6qgb6nBeVQW3dvHgboc19u7T7vyB86zRptt8Ew09ilzyX6ZrK ZlQql5SFAs4OPiEYspvicmabBIxl55NyL2Qv1fmJ/WFrE+rBeKO2SB7iq0ZAOxKlxNID 8EziWdMhA09LyMHZzCuc2wefDlfj5PbKOf/+DFpuwKqza6oIonvW1uiCEWJi3mcY8w8i hEqMt1b4LllyATNYCBhHEO9k0wfW8M+KRmLyIr4z2dXbbkW3NnqdG9+dx2y35AIBHtxf spCg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=Dm0P0EGuKeATfeCtNBK49sionbdZGtbXaMOAetX5rZs=; b=Wnm6Hor4DnfaapxsfjZqrsubBYgtFq89ziqO3PMurJDxxnWy1nBe548IlrfbSxdzs4 gEDhIS3ZuJEltt2zKf0/6EbFBQF2c1RtF94IgGYz6uZmcGnPR17UJdAfEk5TGJ7SK2l5 fftYnxdexxjyWU9f3NVUyWfNsAG8wzch+ye8s/UWxMSaGQa/FwdhtRGaKpd36VM7eCqt NYZ3VpqRqrTPr5nCEW/kWIXrebXeu/nkUYqEwbAFgOW0fe6y2wETnSkEgJctPskYXyQs C6KGHR4R2R0Lg4joODjd7mwPqpcofR6nR0dW3XmtPckibUDGLRVAV6+Q3/yZ3jbEatzf svsA== X-Gm-Message-State: AOAM5300buVKFmup8xBkRJVkJCy8yBvjuZ6g/tJm1Urtdnb3o9wrpbaz skS6bGmqfqaro8MdMtOirTL+iyEBMiU= X-Received: by 2002:a17:90a:df16:: with SMTP id gp22mr2330595pjb.6.1591146843684; Tue, 02 Jun 2020 18:14:03 -0700 (PDT) Received: from localhost.localdomain (174-21-143-238.tukw.qwest.net. [174.21.143.238]) by smtp.gmail.com with ESMTPSA id 3sm290067pfe.85.2020.06.02.18.14.02 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 02 Jun 2020 18:14:02 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v7 34/42] target/arm: Handle TBI for sve scalar + int memory ops Date: Tue, 2 Jun 2020 18:13:09 -0700 Message-Id: <20200603011317.473934-35-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200603011317.473934-1-richard.henderson@linaro.org> References: <20200603011317.473934-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::541; envelope-from=richard.henderson@linaro.org; helo=mail-pg1-x541.google.com X-detected-operating-system: by eggs.gnu.org: No matching host in p0f cache. That's all we know. X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: peter.maydell@linaro.org, qemu-arm@nongnu.org, steplong@quicinc.com Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" We still need to handle tbi for user-only when mte is inactive. Signed-off-by: Richard Henderson --- target/arm/translate-a64.h | 1 + target/arm/translate-a64.c | 2 +- target/arm/translate-sve.c | 6 ++++-- 3 files changed, 6 insertions(+), 3 deletions(-) -- 2.25.1 diff --git a/target/arm/translate-a64.h b/target/arm/translate-a64.h index d0f2e131b3..548414d09f 100644 --- a/target/arm/translate-a64.h +++ b/target/arm/translate-a64.h @@ -40,6 +40,7 @@ TCGv_ptr get_fpstatus_ptr(bool); bool logic_imm_decode_wmask(uint64_t *result, unsigned int immn, unsigned int imms, unsigned int immr); bool sve_access_check(DisasContext *s); +TCGv_i64 clean_data_tbi(DisasContext *s, TCGv_i64 addr); TCGv_i64 gen_mte_check1(DisasContext *s, TCGv_i64 addr, bool is_write, bool tag_checked, int log2_size); TCGv_i64 gen_mte_checkN(DisasContext *s, TCGv_i64 addr, bool is_write, diff --git a/target/arm/translate-a64.c b/target/arm/translate-a64.c index c7deeca86a..394633babd 100644 --- a/target/arm/translate-a64.c +++ b/target/arm/translate-a64.c @@ -215,7 +215,7 @@ static void gen_a64_set_pc(DisasContext *s, TCGv_i64 src) * of the write-back address. */ -static TCGv_i64 clean_data_tbi(DisasContext *s, TCGv_i64 addr) +TCGv_i64 clean_data_tbi(DisasContext *s, TCGv_i64 addr) { TCGv_i64 clean = new_tmp_a64(s); #ifdef CONFIG_USER_ONLY diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index a1dc325104..e7ed48c418 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -4534,9 +4534,8 @@ static void do_mem_zpa(DisasContext *s, int zt, int pg, TCGv_i64 addr, * For e.g. LD4, there are not enough arguments to pass all 4 * registers as pointers, so encode the regno into the data field. * For consistency, do this even for LD1. - * TODO: mte_n check here while callers are updated. */ - if (mte_n && s->mte_active[0]) { + if (s->mte_active[0]) { int msz = dtype_msz(dtype); desc = FIELD_DP32(desc, MTEDESC, MIDX, get_mem_index(s)); @@ -4546,7 +4545,10 @@ static void do_mem_zpa(DisasContext *s, int zt, int pg, TCGv_i64 addr, desc = FIELD_DP32(desc, MTEDESC, ESIZE, 1 << msz); desc = FIELD_DP32(desc, MTEDESC, TSIZE, mte_n << msz); desc <<= SVE_MTEDESC_SHIFT; + } else { + addr = clean_data_tbi(s, addr); } + desc = simd_desc(vsz, vsz, zt | desc); t_desc = tcg_const_i32(desc); t_pg = tcg_temp_new_ptr(); From patchwork Wed Jun 3 01:13:10 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 187441 Delivered-To: patch@linaro.org Received: by 2002:a92:cf06:0:0:0:0:0 with SMTP id c6csp81743ilo; Tue, 2 Jun 2020 18:33:58 -0700 (PDT) X-Google-Smtp-Source: ABdhPJzYkIifTfOAL9wsY81kZ4KkCWztW4luxo+yuReylDZ3ZRcbaKag5mQIxqWtl9QwYwjKHQFn X-Received: by 2002:a25:5387:: with SMTP id h129mr45817747ybb.413.1591148038044; Tue, 02 Jun 2020 18:33:58 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1591148038; cv=none; d=google.com; s=arc-20160816; b=qtV/E0Ni9QQhkYASz2jJ7wVtYkbKMJ7DV7+4uVfBj6Q0XTqWLlSafgwEUrR4JBxplz 3UVFlaw0cL68KZZEbPRxGFmgxoQJNgPiyTbM79gea7JC0X5t3D+ks4Ay4nSAj+Xgpu9a 6vI3Dlm+pZLgixtF+J3S/S9X/H4PWDDwRKQfy4giMSN8q8TTc4rQatU/Ue73+na8A5zw Tmf1L/UjhMB38lrm/zAEhbg/kHwuWKaQhZ5UFz3qmvjjlVH6aMCIhfFgc5RhooKPT45k FKAWaipO4CJ8Dqiyce329XqTwYvyp7LL7yYmueS11Mbse9ZBzBy8yVP/e2lK5W/L0Frb Nbsg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=ivrtMbaTKhkN87Unogxvjpgcle8E0z2zew2IgQNTORs=; b=NRBhhQ7+GLYUpMoCrVIPimzuWmn8r6Mm+Z4TxYKUhLKE6paN29uLFORD5s+lV5CHuT uEApbFqG40uJkLXjmWSmLVmczhnoeBMMLye+b3frueirfzQ/79M+1lYpI7rRIC0PvPae 0rGpe+8Aydl0kCkI55ivpTKtJ0c0QXSxQ5wbgJY9r4Cj+qB1FzKWa14vcNWtIfVr574k lls6X1lguzi7FJ82q0dA7m2c9XfFDPk5eJ5TXzILNRSvze5sxGnpP5qzzjMI7yQPaWDK 6Dptoz2qvB8dDiFJccv0esgEvqNlC1hRFpKWdLWK2x4zIoQd401Xgjb+RKLusPWLD5tY 1sYg== ARC-Authentication-Results: i=1; mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=rxSkVXd0; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id z8si352989ybm.9.2020.06.02.18.33.57 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 02 Jun 2020 18:33:58 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=rxSkVXd0; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:59386 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jgII1-0001iW-Fy for patch@linaro.org; Tue, 02 Jun 2020 21:33:57 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:49044) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jgHyt-00059U-0S for qemu-devel@nongnu.org; Tue, 02 Jun 2020 21:14:11 -0400 Received: from mail-pf1-x434.google.com ([2607:f8b0:4864:20::434]:45276) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1jgHyp-0003gh-WF for qemu-devel@nongnu.org; Tue, 02 Jun 2020 21:14:10 -0400 Received: by mail-pf1-x434.google.com with SMTP id a127so398319pfa.12 for ; Tue, 02 Jun 2020 18:14:07 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=ivrtMbaTKhkN87Unogxvjpgcle8E0z2zew2IgQNTORs=; b=rxSkVXd0vLn2DoSLjIEc/78iqzpPLy3HgoxG/e5kdymIATECI0qBHjbn9jJwgLBo// GiwaF5ntXFyb9B+BLH/ZaHLRHjoA+q3MjblfbmYGynCUto6lZT/YsnuDcqB4wfZNikCr Y8InDHCtK9zXhfD5jK7xbYng+2xR/JIZrh3ML1R61benQ0WQboMgCFT6g/K9OH03Tc92 1d4JFMLG1lPhaWRC9e5vs5pwsAlQbKMwv4w0CdUfz8pUCulpOeb9/og5e4N8Ee4FH2ED OOgXc72e66+tP1hUhW8E9+M4LeewUiqXdNr8+jvHgB3KH6TyUYHUqxjDUQZM50dXHfDK UNPg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=ivrtMbaTKhkN87Unogxvjpgcle8E0z2zew2IgQNTORs=; b=tDSu6/QoYjTEhos/w5kVqhp4MsfKtpaJdEqwQ6GL9iDtUIkWds/CN0lK2/85Gk4MyN pvUseNQz6xci7aYpf0huE27yRNnoG5i1/ZF3iZuRHDucsAZ7d15Ld3m9fxcC8mNQ053x Wp+5MbaYPTmzXOq4ysCNHvByr7P5VU0STbfgbclwTcu2uqgDHZakhRmnVFyOlFJiZfzj k0+cbqIVYXT60R30DkFAn9QnuscWcHtmuZ/k/rh8e6Jon7inukyvtbD2EvlZ8q+9eRpl Vkr4OtZ++Qd7/Ox+oufubVEVADTzE6T5aMBOkzuB8cEfPnHCj73Qaw4312mTb+MOmMqR UUjA== X-Gm-Message-State: AOAM531rVpej0IlD/Yeq9mVMZ0D6ulEFJGOfy2y4XUJSuw7hJ5FOB5yZ Wt8Zyn7XO4SM3Eg9/S95tEQVS0n+NT0= X-Received: by 2002:a62:3381:: with SMTP id z123mr28687664pfz.274.1591146844986; Tue, 02 Jun 2020 18:14:04 -0700 (PDT) Received: from localhost.localdomain (174-21-143-238.tukw.qwest.net. [174.21.143.238]) by smtp.gmail.com with ESMTPSA id 3sm290067pfe.85.2020.06.02.18.14.03 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 02 Jun 2020 18:14:04 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v7 35/42] target/arm: Add mte helpers for sve scatter/gather memory ops Date: Tue, 2 Jun 2020 18:13:10 -0700 Message-Id: <20200603011317.473934-36-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200603011317.473934-1-richard.henderson@linaro.org> References: <20200603011317.473934-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::434; envelope-from=richard.henderson@linaro.org; helo=mail-pf1-x434.google.com X-detected-operating-system: by eggs.gnu.org: No matching host in p0f cache. That's all we know. X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: peter.maydell@linaro.org, qemu-arm@nongnu.org, steplong@quicinc.com Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" Because the elements are non-sequential, we cannot eliminate many tests straight away like we can for sequential operations. But we often have the PTE details handy, so we can test for Tagged. Signed-off-by: Richard Henderson --- target/arm/helper-sve.h | 285 ++++++++++++++++ target/arm/sve_helper.c | 185 +++++++++-- target/arm/translate-sve.c | 650 +++++++++++++++++++++++++------------ 3 files changed, 872 insertions(+), 248 deletions(-) -- 2.25.1 diff --git a/target/arm/helper-sve.h b/target/arm/helper-sve.h index 700ed0e1cb..11d627981d 100644 --- a/target/arm/helper-sve.h +++ b/target/arm/helper-sve.h @@ -1620,6 +1620,115 @@ DEF_HELPER_FLAGS_6(sve_ldsds_le_zd, TCG_CALL_NO_WG, DEF_HELPER_FLAGS_6(sve_ldsds_be_zd, TCG_CALL_NO_WG, void, env, ptr, ptr, ptr, tl, i32) +DEF_HELPER_FLAGS_6(sve_ldbsu_zsu_mte, TCG_CALL_NO_WG, + void, env, ptr, ptr, ptr, tl, i32) +DEF_HELPER_FLAGS_6(sve_ldhsu_le_zsu_mte, TCG_CALL_NO_WG, + void, env, ptr, ptr, ptr, tl, i32) +DEF_HELPER_FLAGS_6(sve_ldhsu_be_zsu_mte, TCG_CALL_NO_WG, + void, env, ptr, ptr, ptr, tl, i32) +DEF_HELPER_FLAGS_6(sve_ldss_le_zsu_mte, TCG_CALL_NO_WG, + void, env, ptr, ptr, ptr, tl, i32) +DEF_HELPER_FLAGS_6(sve_ldss_be_zsu_mte, TCG_CALL_NO_WG, + void, env, ptr, ptr, ptr, tl, i32) +DEF_HELPER_FLAGS_6(sve_ldbss_zsu_mte, TCG_CALL_NO_WG, + void, env, ptr, ptr, ptr, tl, i32) +DEF_HELPER_FLAGS_6(sve_ldhss_le_zsu_mte, TCG_CALL_NO_WG, + void, env, ptr, ptr, ptr, tl, i32) +DEF_HELPER_FLAGS_6(sve_ldhss_be_zsu_mte, TCG_CALL_NO_WG, + void, env, ptr, ptr, ptr, tl, i32) + +DEF_HELPER_FLAGS_6(sve_ldbsu_zss_mte, TCG_CALL_NO_WG, + void, env, ptr, ptr, ptr, tl, i32) +DEF_HELPER_FLAGS_6(sve_ldhsu_le_zss_mte, TCG_CALL_NO_WG, + void, env, ptr, ptr, ptr, tl, i32) +DEF_HELPER_FLAGS_6(sve_ldhsu_be_zss_mte, TCG_CALL_NO_WG, + void, env, ptr, ptr, ptr, tl, i32) +DEF_HELPER_FLAGS_6(sve_ldss_le_zss_mte, TCG_CALL_NO_WG, + void, env, ptr, ptr, ptr, tl, i32) +DEF_HELPER_FLAGS_6(sve_ldss_be_zss_mte, TCG_CALL_NO_WG, + void, env, ptr, ptr, ptr, tl, i32) +DEF_HELPER_FLAGS_6(sve_ldbss_zss_mte, TCG_CALL_NO_WG, + void, env, ptr, ptr, ptr, tl, i32) +DEF_HELPER_FLAGS_6(sve_ldhss_le_zss_mte, TCG_CALL_NO_WG, + void, env, ptr, ptr, ptr, tl, i32) +DEF_HELPER_FLAGS_6(sve_ldhss_be_zss_mte, TCG_CALL_NO_WG, + void, env, ptr, ptr, ptr, tl, i32) + +DEF_HELPER_FLAGS_6(sve_ldbdu_zsu_mte, TCG_CALL_NO_WG, + void, env, ptr, ptr, ptr, tl, i32) +DEF_HELPER_FLAGS_6(sve_ldhdu_le_zsu_mte, TCG_CALL_NO_WG, + void, env, ptr, ptr, ptr, tl, i32) +DEF_HELPER_FLAGS_6(sve_ldhdu_be_zsu_mte, TCG_CALL_NO_WG, + void, env, ptr, ptr, ptr, tl, i32) +DEF_HELPER_FLAGS_6(sve_ldsdu_le_zsu_mte, TCG_CALL_NO_WG, + void, env, ptr, ptr, ptr, tl, i32) +DEF_HELPER_FLAGS_6(sve_ldsdu_be_zsu_mte, TCG_CALL_NO_WG, + void, env, ptr, ptr, ptr, tl, i32) +DEF_HELPER_FLAGS_6(sve_lddd_le_zsu_mte, TCG_CALL_NO_WG, + void, env, ptr, ptr, ptr, tl, i32) +DEF_HELPER_FLAGS_6(sve_lddd_be_zsu_mte, TCG_CALL_NO_WG, + void, env, ptr, ptr, ptr, tl, i32) +DEF_HELPER_FLAGS_6(sve_ldbds_zsu_mte, TCG_CALL_NO_WG, + void, env, ptr, ptr, ptr, tl, i32) +DEF_HELPER_FLAGS_6(sve_ldhds_le_zsu_mte, TCG_CALL_NO_WG, + void, env, ptr, ptr, ptr, tl, i32) +DEF_HELPER_FLAGS_6(sve_ldhds_be_zsu_mte, TCG_CALL_NO_WG, + void, env, ptr, ptr, ptr, tl, i32) +DEF_HELPER_FLAGS_6(sve_ldsds_le_zsu_mte, TCG_CALL_NO_WG, + void, env, ptr, ptr, ptr, tl, i32) +DEF_HELPER_FLAGS_6(sve_ldsds_be_zsu_mte, TCG_CALL_NO_WG, + void, env, ptr, ptr, ptr, tl, i32) + +DEF_HELPER_FLAGS_6(sve_ldbdu_zss_mte, TCG_CALL_NO_WG, + void, env, ptr, ptr, ptr, tl, i32) +DEF_HELPER_FLAGS_6(sve_ldhdu_le_zss_mte, TCG_CALL_NO_WG, + void, env, ptr, ptr, ptr, tl, i32) +DEF_HELPER_FLAGS_6(sve_ldhdu_be_zss_mte, TCG_CALL_NO_WG, + void, env, ptr, ptr, ptr, tl, i32) +DEF_HELPER_FLAGS_6(sve_ldsdu_le_zss_mte, TCG_CALL_NO_WG, + void, env, ptr, ptr, ptr, tl, i32) +DEF_HELPER_FLAGS_6(sve_ldsdu_be_zss_mte, TCG_CALL_NO_WG, + void, env, ptr, ptr, ptr, tl, i32) +DEF_HELPER_FLAGS_6(sve_lddd_le_zss_mte, TCG_CALL_NO_WG, + void, env, ptr, ptr, ptr, tl, i32) +DEF_HELPER_FLAGS_6(sve_lddd_be_zss_mte, TCG_CALL_NO_WG, + void, env, ptr, ptr, ptr, tl, i32) +DEF_HELPER_FLAGS_6(sve_ldbds_zss_mte, TCG_CALL_NO_WG, + void, env, ptr, ptr, ptr, tl, i32) +DEF_HELPER_FLAGS_6(sve_ldhds_le_zss_mte, TCG_CALL_NO_WG, + void, env, ptr, ptr, ptr, tl, i32) +DEF_HELPER_FLAGS_6(sve_ldhds_be_zss_mte, TCG_CALL_NO_WG, + void, env, ptr, ptr, ptr, tl, i32) +DEF_HELPER_FLAGS_6(sve_ldsds_le_zss_mte, TCG_CALL_NO_WG, + void, env, ptr, ptr, ptr, tl, i32) +DEF_HELPER_FLAGS_6(sve_ldsds_be_zss_mte, TCG_CALL_NO_WG, + void, env, ptr, ptr, ptr, tl, i32) + +DEF_HELPER_FLAGS_6(sve_ldbdu_zd_mte, TCG_CALL_NO_WG, + void, env, ptr, ptr, ptr, tl, i32) +DEF_HELPER_FLAGS_6(sve_ldhdu_le_zd_mte, TCG_CALL_NO_WG, + void, env, ptr, ptr, ptr, tl, i32) +DEF_HELPER_FLAGS_6(sve_ldhdu_be_zd_mte, TCG_CALL_NO_WG, + void, env, ptr, ptr, ptr, tl, i32) +DEF_HELPER_FLAGS_6(sve_ldsdu_le_zd_mte, TCG_CALL_NO_WG, + void, env, ptr, ptr, ptr, tl, i32) +DEF_HELPER_FLAGS_6(sve_ldsdu_be_zd_mte, TCG_CALL_NO_WG, + void, env, ptr, ptr, ptr, tl, i32) +DEF_HELPER_FLAGS_6(sve_lddd_le_zd_mte, TCG_CALL_NO_WG, + void, env, ptr, ptr, ptr, tl, i32) +DEF_HELPER_FLAGS_6(sve_lddd_be_zd_mte, TCG_CALL_NO_WG, + void, env, ptr, ptr, ptr, tl, i32) +DEF_HELPER_FLAGS_6(sve_ldbds_zd_mte, TCG_CALL_NO_WG, + void, env, ptr, ptr, ptr, tl, i32) +DEF_HELPER_FLAGS_6(sve_ldhds_le_zd_mte, TCG_CALL_NO_WG, + void, env, ptr, ptr, ptr, tl, i32) +DEF_HELPER_FLAGS_6(sve_ldhds_be_zd_mte, TCG_CALL_NO_WG, + void, env, ptr, ptr, ptr, tl, i32) +DEF_HELPER_FLAGS_6(sve_ldsds_le_zd_mte, TCG_CALL_NO_WG, + void, env, ptr, ptr, ptr, tl, i32) +DEF_HELPER_FLAGS_6(sve_ldsds_be_zd_mte, TCG_CALL_NO_WG, + void, env, ptr, ptr, ptr, tl, i32) + DEF_HELPER_FLAGS_6(sve_ldffbsu_zsu, TCG_CALL_NO_WG, void, env, ptr, ptr, ptr, tl, i32) DEF_HELPER_FLAGS_6(sve_ldffhsu_le_zsu, TCG_CALL_NO_WG, @@ -1729,6 +1838,115 @@ DEF_HELPER_FLAGS_6(sve_ldffsds_le_zd, TCG_CALL_NO_WG, DEF_HELPER_FLAGS_6(sve_ldffsds_be_zd, TCG_CALL_NO_WG, void, env, ptr, ptr, ptr, tl, i32) +DEF_HELPER_FLAGS_6(sve_ldffbsu_zsu_mte, TCG_CALL_NO_WG, + void, env, ptr, ptr, ptr, tl, i32) +DEF_HELPER_FLAGS_6(sve_ldffhsu_le_zsu_mte, TCG_CALL_NO_WG, + void, env, ptr, ptr, ptr, tl, i32) +DEF_HELPER_FLAGS_6(sve_ldffhsu_be_zsu_mte, TCG_CALL_NO_WG, + void, env, ptr, ptr, ptr, tl, i32) +DEF_HELPER_FLAGS_6(sve_ldffss_le_zsu_mte, TCG_CALL_NO_WG, + void, env, ptr, ptr, ptr, tl, i32) +DEF_HELPER_FLAGS_6(sve_ldffss_be_zsu_mte, TCG_CALL_NO_WG, + void, env, ptr, ptr, ptr, tl, i32) +DEF_HELPER_FLAGS_6(sve_ldffbss_zsu_mte, TCG_CALL_NO_WG, + void, env, ptr, ptr, ptr, tl, i32) +DEF_HELPER_FLAGS_6(sve_ldffhss_le_zsu_mte, TCG_CALL_NO_WG, + void, env, ptr, ptr, ptr, tl, i32) +DEF_HELPER_FLAGS_6(sve_ldffhss_be_zsu_mte, TCG_CALL_NO_WG, + void, env, ptr, ptr, ptr, tl, i32) + +DEF_HELPER_FLAGS_6(sve_ldffbsu_zss_mte, TCG_CALL_NO_WG, + void, env, ptr, ptr, ptr, tl, i32) +DEF_HELPER_FLAGS_6(sve_ldffhsu_le_zss_mte, TCG_CALL_NO_WG, + void, env, ptr, ptr, ptr, tl, i32) +DEF_HELPER_FLAGS_6(sve_ldffhsu_be_zss_mte, TCG_CALL_NO_WG, + void, env, ptr, ptr, ptr, tl, i32) +DEF_HELPER_FLAGS_6(sve_ldffss_le_zss_mte, TCG_CALL_NO_WG, + void, env, ptr, ptr, ptr, tl, i32) +DEF_HELPER_FLAGS_6(sve_ldffss_be_zss_mte, TCG_CALL_NO_WG, + void, env, ptr, ptr, ptr, tl, i32) +DEF_HELPER_FLAGS_6(sve_ldffbss_zss_mte, TCG_CALL_NO_WG, + void, env, ptr, ptr, ptr, tl, i32) +DEF_HELPER_FLAGS_6(sve_ldffhss_le_zss_mte, TCG_CALL_NO_WG, + void, env, ptr, ptr, ptr, tl, i32) +DEF_HELPER_FLAGS_6(sve_ldffhss_be_zss_mte, TCG_CALL_NO_WG, + void, env, ptr, ptr, ptr, tl, i32) + +DEF_HELPER_FLAGS_6(sve_ldffbdu_zsu_mte, TCG_CALL_NO_WG, + void, env, ptr, ptr, ptr, tl, i32) +DEF_HELPER_FLAGS_6(sve_ldffhdu_le_zsu_mte, TCG_CALL_NO_WG, + void, env, ptr, ptr, ptr, tl, i32) +DEF_HELPER_FLAGS_6(sve_ldffhdu_be_zsu_mte, TCG_CALL_NO_WG, + void, env, ptr, ptr, ptr, tl, i32) +DEF_HELPER_FLAGS_6(sve_ldffsdu_le_zsu_mte, TCG_CALL_NO_WG, + void, env, ptr, ptr, ptr, tl, i32) +DEF_HELPER_FLAGS_6(sve_ldffsdu_be_zsu_mte, TCG_CALL_NO_WG, + void, env, ptr, ptr, ptr, tl, i32) +DEF_HELPER_FLAGS_6(sve_ldffdd_le_zsu_mte, TCG_CALL_NO_WG, + void, env, ptr, ptr, ptr, tl, i32) +DEF_HELPER_FLAGS_6(sve_ldffdd_be_zsu_mte, TCG_CALL_NO_WG, + void, env, ptr, ptr, ptr, tl, i32) +DEF_HELPER_FLAGS_6(sve_ldffbds_zsu_mte, TCG_CALL_NO_WG, + void, env, ptr, ptr, ptr, tl, i32) +DEF_HELPER_FLAGS_6(sve_ldffhds_le_zsu_mte, TCG_CALL_NO_WG, + void, env, ptr, ptr, ptr, tl, i32) +DEF_HELPER_FLAGS_6(sve_ldffhds_be_zsu_mte, TCG_CALL_NO_WG, + void, env, ptr, ptr, ptr, tl, i32) +DEF_HELPER_FLAGS_6(sve_ldffsds_le_zsu_mte, TCG_CALL_NO_WG, + void, env, ptr, ptr, ptr, tl, i32) +DEF_HELPER_FLAGS_6(sve_ldffsds_be_zsu_mte, TCG_CALL_NO_WG, + void, env, ptr, ptr, ptr, tl, i32) + +DEF_HELPER_FLAGS_6(sve_ldffbdu_zss_mte, TCG_CALL_NO_WG, + void, env, ptr, ptr, ptr, tl, i32) +DEF_HELPER_FLAGS_6(sve_ldffhdu_le_zss_mte, TCG_CALL_NO_WG, + void, env, ptr, ptr, ptr, tl, i32) +DEF_HELPER_FLAGS_6(sve_ldffhdu_be_zss_mte, TCG_CALL_NO_WG, + void, env, ptr, ptr, ptr, tl, i32) +DEF_HELPER_FLAGS_6(sve_ldffsdu_le_zss_mte, TCG_CALL_NO_WG, + void, env, ptr, ptr, ptr, tl, i32) +DEF_HELPER_FLAGS_6(sve_ldffsdu_be_zss_mte, TCG_CALL_NO_WG, + void, env, ptr, ptr, ptr, tl, i32) +DEF_HELPER_FLAGS_6(sve_ldffdd_le_zss_mte, TCG_CALL_NO_WG, + void, env, ptr, ptr, ptr, tl, i32) +DEF_HELPER_FLAGS_6(sve_ldffdd_be_zss_mte, TCG_CALL_NO_WG, + void, env, ptr, ptr, ptr, tl, i32) +DEF_HELPER_FLAGS_6(sve_ldffbds_zss_mte, TCG_CALL_NO_WG, + void, env, ptr, ptr, ptr, tl, i32) +DEF_HELPER_FLAGS_6(sve_ldffhds_le_zss_mte, TCG_CALL_NO_WG, + void, env, ptr, ptr, ptr, tl, i32) +DEF_HELPER_FLAGS_6(sve_ldffhds_be_zss_mte, TCG_CALL_NO_WG, + void, env, ptr, ptr, ptr, tl, i32) +DEF_HELPER_FLAGS_6(sve_ldffsds_le_zss_mte, TCG_CALL_NO_WG, + void, env, ptr, ptr, ptr, tl, i32) +DEF_HELPER_FLAGS_6(sve_ldffsds_be_zss_mte, TCG_CALL_NO_WG, + void, env, ptr, ptr, ptr, tl, i32) + +DEF_HELPER_FLAGS_6(sve_ldffbdu_zd_mte, TCG_CALL_NO_WG, + void, env, ptr, ptr, ptr, tl, i32) +DEF_HELPER_FLAGS_6(sve_ldffhdu_le_zd_mte, TCG_CALL_NO_WG, + void, env, ptr, ptr, ptr, tl, i32) +DEF_HELPER_FLAGS_6(sve_ldffhdu_be_zd_mte, TCG_CALL_NO_WG, + void, env, ptr, ptr, ptr, tl, i32) +DEF_HELPER_FLAGS_6(sve_ldffsdu_le_zd_mte, TCG_CALL_NO_WG, + void, env, ptr, ptr, ptr, tl, i32) +DEF_HELPER_FLAGS_6(sve_ldffsdu_be_zd_mte, TCG_CALL_NO_WG, + void, env, ptr, ptr, ptr, tl, i32) +DEF_HELPER_FLAGS_6(sve_ldffdd_le_zd_mte, TCG_CALL_NO_WG, + void, env, ptr, ptr, ptr, tl, i32) +DEF_HELPER_FLAGS_6(sve_ldffdd_be_zd_mte, TCG_CALL_NO_WG, + void, env, ptr, ptr, ptr, tl, i32) +DEF_HELPER_FLAGS_6(sve_ldffbds_zd_mte, TCG_CALL_NO_WG, + void, env, ptr, ptr, ptr, tl, i32) +DEF_HELPER_FLAGS_6(sve_ldffhds_le_zd_mte, TCG_CALL_NO_WG, + void, env, ptr, ptr, ptr, tl, i32) +DEF_HELPER_FLAGS_6(sve_ldffhds_be_zd_mte, TCG_CALL_NO_WG, + void, env, ptr, ptr, ptr, tl, i32) +DEF_HELPER_FLAGS_6(sve_ldffsds_le_zd_mte, TCG_CALL_NO_WG, + void, env, ptr, ptr, ptr, tl, i32) +DEF_HELPER_FLAGS_6(sve_ldffsds_be_zd_mte, TCG_CALL_NO_WG, + void, env, ptr, ptr, ptr, tl, i32) + DEF_HELPER_FLAGS_6(sve_stbs_zsu, TCG_CALL_NO_WG, void, env, ptr, ptr, ptr, tl, i32) DEF_HELPER_FLAGS_6(sve_sths_le_zsu, TCG_CALL_NO_WG, @@ -1796,4 +2014,71 @@ DEF_HELPER_FLAGS_6(sve_stdd_le_zd, TCG_CALL_NO_WG, DEF_HELPER_FLAGS_6(sve_stdd_be_zd, TCG_CALL_NO_WG, void, env, ptr, ptr, ptr, tl, i32) +DEF_HELPER_FLAGS_6(sve_stbs_zsu_mte, TCG_CALL_NO_WG, + void, env, ptr, ptr, ptr, tl, i32) +DEF_HELPER_FLAGS_6(sve_sths_le_zsu_mte, TCG_CALL_NO_WG, + void, env, ptr, ptr, ptr, tl, i32) +DEF_HELPER_FLAGS_6(sve_sths_be_zsu_mte, TCG_CALL_NO_WG, + void, env, ptr, ptr, ptr, tl, i32) +DEF_HELPER_FLAGS_6(sve_stss_le_zsu_mte, TCG_CALL_NO_WG, + void, env, ptr, ptr, ptr, tl, i32) +DEF_HELPER_FLAGS_6(sve_stss_be_zsu_mte, TCG_CALL_NO_WG, + void, env, ptr, ptr, ptr, tl, i32) + +DEF_HELPER_FLAGS_6(sve_stbs_zss_mte, TCG_CALL_NO_WG, + void, env, ptr, ptr, ptr, tl, i32) +DEF_HELPER_FLAGS_6(sve_sths_le_zss_mte, TCG_CALL_NO_WG, + void, env, ptr, ptr, ptr, tl, i32) +DEF_HELPER_FLAGS_6(sve_sths_be_zss_mte, TCG_CALL_NO_WG, + void, env, ptr, ptr, ptr, tl, i32) +DEF_HELPER_FLAGS_6(sve_stss_le_zss_mte, TCG_CALL_NO_WG, + void, env, ptr, ptr, ptr, tl, i32) +DEF_HELPER_FLAGS_6(sve_stss_be_zss_mte, TCG_CALL_NO_WG, + void, env, ptr, ptr, ptr, tl, i32) + +DEF_HELPER_FLAGS_6(sve_stbd_zsu_mte, TCG_CALL_NO_WG, + void, env, ptr, ptr, ptr, tl, i32) +DEF_HELPER_FLAGS_6(sve_sthd_le_zsu_mte, TCG_CALL_NO_WG, + void, env, ptr, ptr, ptr, tl, i32) +DEF_HELPER_FLAGS_6(sve_sthd_be_zsu_mte, TCG_CALL_NO_WG, + void, env, ptr, ptr, ptr, tl, i32) +DEF_HELPER_FLAGS_6(sve_stsd_le_zsu_mte, TCG_CALL_NO_WG, + void, env, ptr, ptr, ptr, tl, i32) +DEF_HELPER_FLAGS_6(sve_stsd_be_zsu_mte, TCG_CALL_NO_WG, + void, env, ptr, ptr, ptr, tl, i32) +DEF_HELPER_FLAGS_6(sve_stdd_le_zsu_mte, TCG_CALL_NO_WG, + void, env, ptr, ptr, ptr, tl, i32) +DEF_HELPER_FLAGS_6(sve_stdd_be_zsu_mte, TCG_CALL_NO_WG, + void, env, ptr, ptr, ptr, tl, i32) + +DEF_HELPER_FLAGS_6(sve_stbd_zss_mte, TCG_CALL_NO_WG, + void, env, ptr, ptr, ptr, tl, i32) +DEF_HELPER_FLAGS_6(sve_sthd_le_zss_mte, TCG_CALL_NO_WG, + void, env, ptr, ptr, ptr, tl, i32) +DEF_HELPER_FLAGS_6(sve_sthd_be_zss_mte, TCG_CALL_NO_WG, + void, env, ptr, ptr, ptr, tl, i32) +DEF_HELPER_FLAGS_6(sve_stsd_le_zss_mte, TCG_CALL_NO_WG, + void, env, ptr, ptr, ptr, tl, i32) +DEF_HELPER_FLAGS_6(sve_stsd_be_zss_mte, TCG_CALL_NO_WG, + void, env, ptr, ptr, ptr, tl, i32) +DEF_HELPER_FLAGS_6(sve_stdd_le_zss_mte, TCG_CALL_NO_WG, + void, env, ptr, ptr, ptr, tl, i32) +DEF_HELPER_FLAGS_6(sve_stdd_be_zss_mte, TCG_CALL_NO_WG, + void, env, ptr, ptr, ptr, tl, i32) + +DEF_HELPER_FLAGS_6(sve_stbd_zd_mte, TCG_CALL_NO_WG, + void, env, ptr, ptr, ptr, tl, i32) +DEF_HELPER_FLAGS_6(sve_sthd_le_zd_mte, TCG_CALL_NO_WG, + void, env, ptr, ptr, ptr, tl, i32) +DEF_HELPER_FLAGS_6(sve_sthd_be_zd_mte, TCG_CALL_NO_WG, + void, env, ptr, ptr, ptr, tl, i32) +DEF_HELPER_FLAGS_6(sve_stsd_le_zd_mte, TCG_CALL_NO_WG, + void, env, ptr, ptr, ptr, tl, i32) +DEF_HELPER_FLAGS_6(sve_stsd_be_zd_mte, TCG_CALL_NO_WG, + void, env, ptr, ptr, ptr, tl, i32) +DEF_HELPER_FLAGS_6(sve_stdd_le_zd_mte, TCG_CALL_NO_WG, + void, env, ptr, ptr, ptr, tl, i32) +DEF_HELPER_FLAGS_6(sve_stdd_be_zd_mte, TCG_CALL_NO_WG, + void, env, ptr, ptr, ptr, tl, i32) + DEF_HELPER_FLAGS_4(sve2_pmull_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) diff --git a/target/arm/sve_helper.c b/target/arm/sve_helper.c index 85bf8e5de3..2d2aa14b43 100644 --- a/target/arm/sve_helper.c +++ b/target/arm/sve_helper.c @@ -5482,7 +5482,8 @@ static target_ulong off_zd_d(void *reg, intptr_t reg_ofs) static inline QEMU_ALWAYS_INLINE void sve_ld1_z(CPUARMState *env, void *vd, uint64_t *vg, void *vm, target_ulong base, uint32_t desc, uintptr_t retaddr, - int esize, int msize, zreg_off_fn *off_fn, + uint32_t mtedesc, int esize, int msize, + zreg_off_fn *off_fn, sve_ldst1_host_fn *host_fn, sve_ldst1_tlb_fn *tlb_fn) { @@ -5510,7 +5511,9 @@ void sve_ld1_z(CPUARMState *env, void *vd, uint64_t *vg, void *vm, cpu_check_watchpoint(env_cpu(env), addr, msize, info.attrs, BP_MEM_READ, retaddr); } - /* TODO: MTE check */ + if (mtedesc && info.attrs.target_tlb_bit1) { + mte_check1(env, mtedesc, addr, retaddr); + } host_fn(&scratch, reg_off, info.host); } else { /* Element crosses the page boundary. */ @@ -5521,7 +5524,9 @@ void sve_ld1_z(CPUARMState *env, void *vd, uint64_t *vg, void *vm, msize, info.attrs, BP_MEM_READ, retaddr); } - /* TODO: MTE check */ + if (mtedesc && info.attrs.target_tlb_bit1) { + mte_check1(env, mtedesc, addr, retaddr); + } tlb_fn(env, &scratch, reg_off, addr, retaddr); } } @@ -5534,20 +5539,53 @@ void sve_ld1_z(CPUARMState *env, void *vd, uint64_t *vg, void *vm, memcpy(vd, &scratch, reg_max); } +static inline QEMU_ALWAYS_INLINE +void sve_ld1_z_mte(CPUARMState *env, void *vd, uint64_t *vg, void *vm, + target_ulong base, uint32_t desc, uintptr_t retaddr, + int esize, int msize, zreg_off_fn *off_fn, + sve_ldst1_host_fn *host_fn, + sve_ldst1_tlb_fn *tlb_fn) +{ + uint32_t mtedesc = desc >> (SIMD_DATA_SHIFT + SVE_MTEDESC_SHIFT); + /* Remove mtedesc from the normal sve descriptor. */ + desc = extract32(desc, 0, SIMD_DATA_SHIFT + SVE_MTEDESC_SHIFT); + + /* + * ??? TODO: For the 32-bit offset extractions, base + ofs cannot + * offset base entirely over the address space hole to change the + * pointer tag, or change the bit55 selector. So we could here + * examine TBI + TCMA like we do for sve_ldN_r_mte(). + */ + sve_ld1_z(env, vd, vg, vm, base, desc, retaddr, mtedesc, + esize, msize, off_fn, host_fn, tlb_fn); +} + #define DO_LD1_ZPZ_S(MEM, OFS, MSZ) \ void HELPER(sve_ld##MEM##_##OFS)(CPUARMState *env, void *vd, void *vg, \ void *vm, target_ulong base, uint32_t desc) \ { \ - sve_ld1_z(env, vd, vg, vm, base, desc, GETPC(), 4, 1 << MSZ, \ + sve_ld1_z(env, vd, vg, vm, base, desc, GETPC(), 0, 4, 1 << MSZ, \ off_##OFS##_s, sve_ld1##MEM##_host, sve_ld1##MEM##_tlb); \ +} \ +void HELPER(sve_ld##MEM##_##OFS##_mte)(CPUARMState *env, void *vd, void *vg, \ + void *vm, target_ulong base, uint32_t desc) \ +{ \ + sve_ld1_z_mte(env, vd, vg, vm, base, desc, GETPC(), 4, 1 << MSZ, \ + off_##OFS##_s, sve_ld1##MEM##_host, sve_ld1##MEM##_tlb); \ } #define DO_LD1_ZPZ_D(MEM, OFS, MSZ) \ void HELPER(sve_ld##MEM##_##OFS)(CPUARMState *env, void *vd, void *vg, \ void *vm, target_ulong base, uint32_t desc) \ { \ - sve_ld1_z(env, vd, vg, vm, base, desc, GETPC(), 8, 1 << MSZ, \ + sve_ld1_z(env, vd, vg, vm, base, desc, GETPC(), 0, 8, 1 << MSZ, \ off_##OFS##_d, sve_ld1##MEM##_host, sve_ld1##MEM##_tlb); \ +} \ +void HELPER(sve_ld##MEM##_##OFS##_mte)(CPUARMState *env, void *vd, void *vg, \ + void *vm, target_ulong base, uint32_t desc) \ +{ \ + sve_ld1_z_mte(env, vd, vg, vm, base, desc, GETPC(), 8, 1 << MSZ, \ + off_##OFS##_d, sve_ld1##MEM##_host, sve_ld1##MEM##_tlb); \ } DO_LD1_ZPZ_S(bsu, zsu, MO_8) @@ -5626,7 +5664,8 @@ DO_LD1_ZPZ_D(dd_be, zd, MO_64) static inline QEMU_ALWAYS_INLINE void sve_ldff1_z(CPUARMState *env, void *vd, uint64_t *vg, void *vm, target_ulong base, uint32_t desc, uintptr_t retaddr, - const int esz, const int msz, zreg_off_fn *off_fn, + uint32_t mtedesc, const int esz, const int msz, + zreg_off_fn *off_fn, sve_ldst1_host_fn *host_fn, sve_ldst1_tlb_fn *tlb_fn) { @@ -5651,6 +5690,9 @@ void sve_ldff1_z(CPUARMState *env, void *vd, uint64_t *vg, void *vm, * Probe the first element, allowing faults. */ addr = base + (off_fn(vm, reg_off) << scale); + if (mtedesc) { + mte_check1(env, mtedesc, addr, retaddr); + } tlb_fn(env, vd, reg_off, addr, retaddr); /* After any fault, zero the other elements. */ @@ -5683,7 +5725,11 @@ void sve_ldff1_z(CPUARMState *env, void *vd, uint64_t *vg, void *vm, (env_cpu(env), addr, msize) & BP_MEM_READ)) { goto fault; } - /* TODO: MTE check. */ + if (mtedesc && + info.attrs.target_tlb_bit1 && + !mte_probe1(env, mtedesc, addr, retaddr)) { + goto fault; + } host_fn(vd, reg_off, info.host); } @@ -5696,20 +5742,58 @@ void sve_ldff1_z(CPUARMState *env, void *vd, uint64_t *vg, void *vm, record_fault(env, reg_off, reg_max); } -#define DO_LDFF1_ZPZ_S(MEM, OFS, MSZ) \ -void HELPER(sve_ldff##MEM##_##OFS)(CPUARMState *env, void *vd, void *vg, \ - void *vm, target_ulong base, uint32_t desc) \ -{ \ - sve_ldff1_z(env, vd, vg, vm, base, desc, GETPC(), MO_32, MSZ, \ - off_##OFS##_s, sve_ld1##MEM##_host, sve_ld1##MEM##_tlb); \ +static inline QEMU_ALWAYS_INLINE +void sve_ldff1_z_mte(CPUARMState *env, void *vd, uint64_t *vg, void *vm, + target_ulong base, uint32_t desc, uintptr_t retaddr, + const int esz, const int msz, + zreg_off_fn *off_fn, + sve_ldst1_host_fn *host_fn, + sve_ldst1_tlb_fn *tlb_fn) +{ + uint32_t mtedesc = desc >> (SIMD_DATA_SHIFT + SVE_MTEDESC_SHIFT); + /* Remove mtedesc from the normal sve descriptor. */ + desc = extract32(desc, 0, SIMD_DATA_SHIFT + SVE_MTEDESC_SHIFT); + + /* + * ??? TODO: For the 32-bit offset extractions, base + ofs cannot + * offset base entirely over the address space hole to change the + * pointer tag, or change the bit55 selector. So we could here + * examine TBI + TCMA like we do for sve_ldN_r_mte(). + */ + sve_ldff1_z(env, vd, vg, vm, base, desc, retaddr, mtedesc, + esz, msz, off_fn, host_fn, tlb_fn); } -#define DO_LDFF1_ZPZ_D(MEM, OFS, MSZ) \ -void HELPER(sve_ldff##MEM##_##OFS)(CPUARMState *env, void *vd, void *vg, \ - void *vm, target_ulong base, uint32_t desc) \ -{ \ - sve_ldff1_z(env, vd, vg, vm, base, desc, GETPC(), MO_64, MSZ, \ - off_##OFS##_d, sve_ld1##MEM##_host, sve_ld1##MEM##_tlb); \ +#define DO_LDFF1_ZPZ_S(MEM, OFS, MSZ) \ +void HELPER(sve_ldff##MEM##_##OFS) \ + (CPUARMState *env, void *vd, void *vg, \ + void *vm, target_ulong base, uint32_t desc) \ +{ \ + sve_ldff1_z(env, vd, vg, vm, base, desc, GETPC(), 0, MO_32, MSZ, \ + off_##OFS##_s, sve_ld1##MEM##_host, sve_ld1##MEM##_tlb); \ +} \ +void HELPER(sve_ldff##MEM##_##OFS##_mte) \ + (CPUARMState *env, void *vd, void *vg, \ + void *vm, target_ulong base, uint32_t desc) \ +{ \ + sve_ldff1_z_mte(env, vd, vg, vm, base, desc, GETPC(), MO_32, MSZ, \ + off_##OFS##_s, sve_ld1##MEM##_host, sve_ld1##MEM##_tlb); \ +} + +#define DO_LDFF1_ZPZ_D(MEM, OFS, MSZ) \ +void HELPER(sve_ldff##MEM##_##OFS) \ + (CPUARMState *env, void *vd, void *vg, \ + void *vm, target_ulong base, uint32_t desc) \ +{ \ + sve_ldff1_z(env, vd, vg, vm, base, desc, GETPC(), 0, MO_64, MSZ, \ + off_##OFS##_d, sve_ld1##MEM##_host, sve_ld1##MEM##_tlb); \ +} \ +void HELPER(sve_ldff##MEM##_##OFS##_mte) \ + (CPUARMState *env, void *vd, void *vg, \ + void *vm, target_ulong base, uint32_t desc) \ +{ \ + sve_ldff1_z_mte(env, vd, vg, vm, base, desc, GETPC(), MO_64, MSZ, \ + off_##OFS##_d, sve_ld1##MEM##_host, sve_ld1##MEM##_tlb); \ } DO_LDFF1_ZPZ_S(bsu, zsu, MO_8) @@ -5781,7 +5865,8 @@ DO_LDFF1_ZPZ_D(dd_be, zd, MO_64) static inline QEMU_ALWAYS_INLINE void sve_st1_z(CPUARMState *env, void *vd, uint64_t *vg, void *vm, target_ulong base, uint32_t desc, uintptr_t retaddr, - int esize, int msize, zreg_off_fn *off_fn, + uint32_t mtedesc, int esize, int msize, + zreg_off_fn *off_fn, sve_ldst1_host_fn *host_fn, sve_ldst1_tlb_fn *tlb_fn) { @@ -5825,7 +5910,10 @@ void sve_st1_z(CPUARMState *env, void *vd, uint64_t *vg, void *vm, cpu_check_watchpoint(env_cpu(env), addr, msize, info.attrs, BP_MEM_WRITE, retaddr); } - /* TODO: MTE check. */ + + if (mtedesc && info.attrs.target_tlb_bit1) { + mte_check1(env, mtedesc, addr, retaddr); + } } i += 1; reg_off += esize; @@ -5855,20 +5943,53 @@ void sve_st1_z(CPUARMState *env, void *vd, uint64_t *vg, void *vm, } while (reg_off < reg_max); } -#define DO_ST1_ZPZ_S(MEM, OFS, MSZ) \ -void HELPER(sve_st##MEM##_##OFS)(CPUARMState *env, void *vd, void *vg, \ - void *vm, target_ulong base, uint32_t desc) \ -{ \ - sve_st1_z(env, vd, vg, vm, base, desc, GETPC(), 4, 1 << MSZ, \ - off_##OFS##_s, sve_st1##MEM##_host, sve_st1##MEM##_tlb); \ +static inline QEMU_ALWAYS_INLINE +void sve_st1_z_mte(CPUARMState *env, void *vd, uint64_t *vg, void *vm, + target_ulong base, uint32_t desc, uintptr_t retaddr, + int esize, int msize, zreg_off_fn *off_fn, + sve_ldst1_host_fn *host_fn, + sve_ldst1_tlb_fn *tlb_fn) +{ + uint32_t mtedesc = desc >> (SIMD_DATA_SHIFT + SVE_MTEDESC_SHIFT); + /* Remove mtedesc from the normal sve descriptor. */ + desc = extract32(desc, 0, SIMD_DATA_SHIFT + SVE_MTEDESC_SHIFT); + + /* + * ??? TODO: For the 32-bit offset extractions, base + ofs cannot + * offset base entirely over the address space hole to change the + * pointer tag, or change the bit55 selector. So we could here + * examine TBI + TCMA like we do for sve_ldN_r_mte(). + */ + sve_st1_z(env, vd, vg, vm, base, desc, retaddr, mtedesc, + esize, msize, off_fn, host_fn, tlb_fn); } -#define DO_ST1_ZPZ_D(MEM, OFS, MSZ) \ -void HELPER(sve_st##MEM##_##OFS)(CPUARMState *env, void *vd, void *vg, \ +#define DO_ST1_ZPZ_S(MEM, OFS, MSZ) \ +void HELPER(sve_st##MEM##_##OFS)(CPUARMState *env, void *vd, void *vg, \ void *vm, target_ulong base, uint32_t desc) \ -{ \ - sve_st1_z(env, vd, vg, vm, base, desc, GETPC(), 8, 1 << MSZ, \ - off_##OFS##_d, sve_st1##MEM##_host, sve_st1##MEM##_tlb); \ +{ \ + sve_st1_z(env, vd, vg, vm, base, desc, GETPC(), 0, 4, 1 << MSZ, \ + off_##OFS##_s, sve_st1##MEM##_host, sve_st1##MEM##_tlb); \ +} \ +void HELPER(sve_st##MEM##_##OFS##_mte)(CPUARMState *env, void *vd, void *vg, \ + void *vm, target_ulong base, uint32_t desc) \ +{ \ + sve_st1_z_mte(env, vd, vg, vm, base, desc, GETPC(), 4, 1 << MSZ, \ + off_##OFS##_s, sve_st1##MEM##_host, sve_st1##MEM##_tlb); \ +} + +#define DO_ST1_ZPZ_D(MEM, OFS, MSZ) \ +void HELPER(sve_st##MEM##_##OFS)(CPUARMState *env, void *vd, void *vg, \ + void *vm, target_ulong base, uint32_t desc) \ +{ \ + sve_st1_z(env, vd, vg, vm, base, desc, GETPC(), 0, 8, 1 << MSZ, \ + off_##OFS##_d, sve_st1##MEM##_host, sve_st1##MEM##_tlb); \ +} \ +void HELPER(sve_st##MEM##_##OFS##_mte)(CPUARMState *env, void *vd, void *vg, \ + void *vm, target_ulong base, uint32_t desc) \ +{ \ + sve_st1_z_mte(env, vd, vg, vm, base, desc, GETPC(), 8, 1 << MSZ, \ + off_##OFS##_d, sve_st1##MEM##_host, sve_st1##MEM##_tlb); \ } DO_ST1_ZPZ_S(bs, zsu, MO_8) diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index e7ed48c418..9fe649b592 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -5206,7 +5206,7 @@ static bool trans_ST_zpri(DisasContext *s, arg_rpri_store *a) */ static void do_mem_zpz(DisasContext *s, int zt, int pg, int zm, - int scale, TCGv_i64 scalar, int msz, + int scale, TCGv_i64 scalar, int msz, bool is_write, gen_helper_gvec_mem_scatter *fn) { unsigned vsz = vec_full_reg_size(s); @@ -5214,8 +5214,16 @@ static void do_mem_zpz(DisasContext *s, int zt, int pg, int zm, TCGv_ptr t_pg = tcg_temp_new_ptr(); TCGv_ptr t_zt = tcg_temp_new_ptr(); TCGv_i32 t_desc; - int desc; + int desc = 0; + if (s->mte_active[0]) { + desc = FIELD_DP32(desc, MTEDESC, MIDX, get_mem_index(s)); + desc = FIELD_DP32(desc, MTEDESC, TBI, s->tbid); + desc = FIELD_DP32(desc, MTEDESC, TCMA, s->tcma); + desc = FIELD_DP32(desc, MTEDESC, WRITE, is_write); + desc = FIELD_DP32(desc, MTEDESC, ESIZE, 1 << msz); + desc <<= SVE_MTEDESC_SHIFT; + } desc = simd_desc(vsz, vsz, scale); t_desc = tcg_const_i32(desc); @@ -5230,176 +5238,339 @@ static void do_mem_zpz(DisasContext *s, int zt, int pg, int zm, tcg_temp_free_i32(t_desc); } -/* Indexed by [be][ff][xs][u][msz]. */ -static gen_helper_gvec_mem_scatter * const gather_load_fn32[2][2][2][2][3] = { - /* Little-endian */ - { { { { gen_helper_sve_ldbss_zsu, - gen_helper_sve_ldhss_le_zsu, - NULL, }, - { gen_helper_sve_ldbsu_zsu, - gen_helper_sve_ldhsu_le_zsu, - gen_helper_sve_ldss_le_zsu, } }, - { { gen_helper_sve_ldbss_zss, - gen_helper_sve_ldhss_le_zss, - NULL, }, - { gen_helper_sve_ldbsu_zss, - gen_helper_sve_ldhsu_le_zss, - gen_helper_sve_ldss_le_zss, } } }, +/* Indexed by [mte][be][ff][xs][u][msz]. */ +static gen_helper_gvec_mem_scatter * const +gather_load_fn32[2][2][2][2][2][3] = { + { /* MTE Inactive */ + { /* Little-endian */ + { { { gen_helper_sve_ldbss_zsu, + gen_helper_sve_ldhss_le_zsu, + NULL, }, + { gen_helper_sve_ldbsu_zsu, + gen_helper_sve_ldhsu_le_zsu, + gen_helper_sve_ldss_le_zsu, } }, + { { gen_helper_sve_ldbss_zss, + gen_helper_sve_ldhss_le_zss, + NULL, }, + { gen_helper_sve_ldbsu_zss, + gen_helper_sve_ldhsu_le_zss, + gen_helper_sve_ldss_le_zss, } } }, - /* First-fault */ - { { { gen_helper_sve_ldffbss_zsu, - gen_helper_sve_ldffhss_le_zsu, - NULL, }, - { gen_helper_sve_ldffbsu_zsu, - gen_helper_sve_ldffhsu_le_zsu, - gen_helper_sve_ldffss_le_zsu, } }, - { { gen_helper_sve_ldffbss_zss, - gen_helper_sve_ldffhss_le_zss, - NULL, }, - { gen_helper_sve_ldffbsu_zss, - gen_helper_sve_ldffhsu_le_zss, - gen_helper_sve_ldffss_le_zss, } } } }, + /* First-fault */ + { { { gen_helper_sve_ldffbss_zsu, + gen_helper_sve_ldffhss_le_zsu, + NULL, }, + { gen_helper_sve_ldffbsu_zsu, + gen_helper_sve_ldffhsu_le_zsu, + gen_helper_sve_ldffss_le_zsu, } }, + { { gen_helper_sve_ldffbss_zss, + gen_helper_sve_ldffhss_le_zss, + NULL, }, + { gen_helper_sve_ldffbsu_zss, + gen_helper_sve_ldffhsu_le_zss, + gen_helper_sve_ldffss_le_zss, } } } }, - /* Big-endian */ - { { { { gen_helper_sve_ldbss_zsu, - gen_helper_sve_ldhss_be_zsu, - NULL, }, - { gen_helper_sve_ldbsu_zsu, - gen_helper_sve_ldhsu_be_zsu, - gen_helper_sve_ldss_be_zsu, } }, - { { gen_helper_sve_ldbss_zss, - gen_helper_sve_ldhss_be_zss, - NULL, }, - { gen_helper_sve_ldbsu_zss, - gen_helper_sve_ldhsu_be_zss, - gen_helper_sve_ldss_be_zss, } } }, + { /* Big-endian */ + { { { gen_helper_sve_ldbss_zsu, + gen_helper_sve_ldhss_be_zsu, + NULL, }, + { gen_helper_sve_ldbsu_zsu, + gen_helper_sve_ldhsu_be_zsu, + gen_helper_sve_ldss_be_zsu, } }, + { { gen_helper_sve_ldbss_zss, + gen_helper_sve_ldhss_be_zss, + NULL, }, + { gen_helper_sve_ldbsu_zss, + gen_helper_sve_ldhsu_be_zss, + gen_helper_sve_ldss_be_zss, } } }, - /* First-fault */ - { { { gen_helper_sve_ldffbss_zsu, - gen_helper_sve_ldffhss_be_zsu, - NULL, }, - { gen_helper_sve_ldffbsu_zsu, - gen_helper_sve_ldffhsu_be_zsu, - gen_helper_sve_ldffss_be_zsu, } }, - { { gen_helper_sve_ldffbss_zss, - gen_helper_sve_ldffhss_be_zss, - NULL, }, - { gen_helper_sve_ldffbsu_zss, - gen_helper_sve_ldffhsu_be_zss, - gen_helper_sve_ldffss_be_zss, } } } }, + /* First-fault */ + { { { gen_helper_sve_ldffbss_zsu, + gen_helper_sve_ldffhss_be_zsu, + NULL, }, + { gen_helper_sve_ldffbsu_zsu, + gen_helper_sve_ldffhsu_be_zsu, + gen_helper_sve_ldffss_be_zsu, } }, + { { gen_helper_sve_ldffbss_zss, + gen_helper_sve_ldffhss_be_zss, + NULL, }, + { gen_helper_sve_ldffbsu_zss, + gen_helper_sve_ldffhsu_be_zss, + gen_helper_sve_ldffss_be_zss, } } } } }, + { /* MTE Active */ + { /* Little-endian */ + { { { gen_helper_sve_ldbss_zsu_mte, + gen_helper_sve_ldhss_le_zsu_mte, + NULL, }, + { gen_helper_sve_ldbsu_zsu_mte, + gen_helper_sve_ldhsu_le_zsu_mte, + gen_helper_sve_ldss_le_zsu_mte, } }, + { { gen_helper_sve_ldbss_zss_mte, + gen_helper_sve_ldhss_le_zss_mte, + NULL, }, + { gen_helper_sve_ldbsu_zss_mte, + gen_helper_sve_ldhsu_le_zss_mte, + gen_helper_sve_ldss_le_zss_mte, } } }, + + /* First-fault */ + { { { gen_helper_sve_ldffbss_zsu_mte, + gen_helper_sve_ldffhss_le_zsu_mte, + NULL, }, + { gen_helper_sve_ldffbsu_zsu_mte, + gen_helper_sve_ldffhsu_le_zsu_mte, + gen_helper_sve_ldffss_le_zsu_mte, } }, + { { gen_helper_sve_ldffbss_zss_mte, + gen_helper_sve_ldffhss_le_zss_mte, + NULL, }, + { gen_helper_sve_ldffbsu_zss_mte, + gen_helper_sve_ldffhsu_le_zss_mte, + gen_helper_sve_ldffss_le_zss_mte, } } } }, + + { /* Big-endian */ + { { { gen_helper_sve_ldbss_zsu_mte, + gen_helper_sve_ldhss_be_zsu_mte, + NULL, }, + { gen_helper_sve_ldbsu_zsu_mte, + gen_helper_sve_ldhsu_be_zsu_mte, + gen_helper_sve_ldss_be_zsu_mte, } }, + { { gen_helper_sve_ldbss_zss_mte, + gen_helper_sve_ldhss_be_zss_mte, + NULL, }, + { gen_helper_sve_ldbsu_zss_mte, + gen_helper_sve_ldhsu_be_zss_mte, + gen_helper_sve_ldss_be_zss_mte, } } }, + + /* First-fault */ + { { { gen_helper_sve_ldffbss_zsu_mte, + gen_helper_sve_ldffhss_be_zsu_mte, + NULL, }, + { gen_helper_sve_ldffbsu_zsu_mte, + gen_helper_sve_ldffhsu_be_zsu_mte, + gen_helper_sve_ldffss_be_zsu_mte, } }, + { { gen_helper_sve_ldffbss_zss_mte, + gen_helper_sve_ldffhss_be_zss_mte, + NULL, }, + { gen_helper_sve_ldffbsu_zss_mte, + gen_helper_sve_ldffhsu_be_zss_mte, + gen_helper_sve_ldffss_be_zss_mte, } } } } }, }; /* Note that we overload xs=2 to indicate 64-bit offset. */ -static gen_helper_gvec_mem_scatter * const gather_load_fn64[2][2][3][2][4] = { - /* Little-endian */ - { { { { gen_helper_sve_ldbds_zsu, - gen_helper_sve_ldhds_le_zsu, - gen_helper_sve_ldsds_le_zsu, - NULL, }, - { gen_helper_sve_ldbdu_zsu, - gen_helper_sve_ldhdu_le_zsu, - gen_helper_sve_ldsdu_le_zsu, - gen_helper_sve_lddd_le_zsu, } }, - { { gen_helper_sve_ldbds_zss, - gen_helper_sve_ldhds_le_zss, - gen_helper_sve_ldsds_le_zss, - NULL, }, - { gen_helper_sve_ldbdu_zss, - gen_helper_sve_ldhdu_le_zss, - gen_helper_sve_ldsdu_le_zss, - gen_helper_sve_lddd_le_zss, } }, - { { gen_helper_sve_ldbds_zd, - gen_helper_sve_ldhds_le_zd, - gen_helper_sve_ldsds_le_zd, - NULL, }, - { gen_helper_sve_ldbdu_zd, - gen_helper_sve_ldhdu_le_zd, - gen_helper_sve_ldsdu_le_zd, - gen_helper_sve_lddd_le_zd, } } }, +static gen_helper_gvec_mem_scatter * const +gather_load_fn64[2][2][2][3][2][4] = { + { /* MTE Inactive */ + { /* Little-endian */ + { { { gen_helper_sve_ldbds_zsu, + gen_helper_sve_ldhds_le_zsu, + gen_helper_sve_ldsds_le_zsu, + NULL, }, + { gen_helper_sve_ldbdu_zsu, + gen_helper_sve_ldhdu_le_zsu, + gen_helper_sve_ldsdu_le_zsu, + gen_helper_sve_lddd_le_zsu, } }, + { { gen_helper_sve_ldbds_zss, + gen_helper_sve_ldhds_le_zss, + gen_helper_sve_ldsds_le_zss, + NULL, }, + { gen_helper_sve_ldbdu_zss, + gen_helper_sve_ldhdu_le_zss, + gen_helper_sve_ldsdu_le_zss, + gen_helper_sve_lddd_le_zss, } }, + { { gen_helper_sve_ldbds_zd, + gen_helper_sve_ldhds_le_zd, + gen_helper_sve_ldsds_le_zd, + NULL, }, + { gen_helper_sve_ldbdu_zd, + gen_helper_sve_ldhdu_le_zd, + gen_helper_sve_ldsdu_le_zd, + gen_helper_sve_lddd_le_zd, } } }, - /* First-fault */ - { { { gen_helper_sve_ldffbds_zsu, - gen_helper_sve_ldffhds_le_zsu, - gen_helper_sve_ldffsds_le_zsu, - NULL, }, - { gen_helper_sve_ldffbdu_zsu, - gen_helper_sve_ldffhdu_le_zsu, - gen_helper_sve_ldffsdu_le_zsu, - gen_helper_sve_ldffdd_le_zsu, } }, - { { gen_helper_sve_ldffbds_zss, - gen_helper_sve_ldffhds_le_zss, - gen_helper_sve_ldffsds_le_zss, - NULL, }, - { gen_helper_sve_ldffbdu_zss, - gen_helper_sve_ldffhdu_le_zss, - gen_helper_sve_ldffsdu_le_zss, - gen_helper_sve_ldffdd_le_zss, } }, - { { gen_helper_sve_ldffbds_zd, - gen_helper_sve_ldffhds_le_zd, - gen_helper_sve_ldffsds_le_zd, - NULL, }, - { gen_helper_sve_ldffbdu_zd, - gen_helper_sve_ldffhdu_le_zd, - gen_helper_sve_ldffsdu_le_zd, - gen_helper_sve_ldffdd_le_zd, } } } }, + /* First-fault */ + { { { gen_helper_sve_ldffbds_zsu, + gen_helper_sve_ldffhds_le_zsu, + gen_helper_sve_ldffsds_le_zsu, + NULL, }, + { gen_helper_sve_ldffbdu_zsu, + gen_helper_sve_ldffhdu_le_zsu, + gen_helper_sve_ldffsdu_le_zsu, + gen_helper_sve_ldffdd_le_zsu, } }, + { { gen_helper_sve_ldffbds_zss, + gen_helper_sve_ldffhds_le_zss, + gen_helper_sve_ldffsds_le_zss, + NULL, }, + { gen_helper_sve_ldffbdu_zss, + gen_helper_sve_ldffhdu_le_zss, + gen_helper_sve_ldffsdu_le_zss, + gen_helper_sve_ldffdd_le_zss, } }, + { { gen_helper_sve_ldffbds_zd, + gen_helper_sve_ldffhds_le_zd, + gen_helper_sve_ldffsds_le_zd, + NULL, }, + { gen_helper_sve_ldffbdu_zd, + gen_helper_sve_ldffhdu_le_zd, + gen_helper_sve_ldffsdu_le_zd, + gen_helper_sve_ldffdd_le_zd, } } } }, + { /* Big-endian */ + { { { gen_helper_sve_ldbds_zsu, + gen_helper_sve_ldhds_be_zsu, + gen_helper_sve_ldsds_be_zsu, + NULL, }, + { gen_helper_sve_ldbdu_zsu, + gen_helper_sve_ldhdu_be_zsu, + gen_helper_sve_ldsdu_be_zsu, + gen_helper_sve_lddd_be_zsu, } }, + { { gen_helper_sve_ldbds_zss, + gen_helper_sve_ldhds_be_zss, + gen_helper_sve_ldsds_be_zss, + NULL, }, + { gen_helper_sve_ldbdu_zss, + gen_helper_sve_ldhdu_be_zss, + gen_helper_sve_ldsdu_be_zss, + gen_helper_sve_lddd_be_zss, } }, + { { gen_helper_sve_ldbds_zd, + gen_helper_sve_ldhds_be_zd, + gen_helper_sve_ldsds_be_zd, + NULL, }, + { gen_helper_sve_ldbdu_zd, + gen_helper_sve_ldhdu_be_zd, + gen_helper_sve_ldsdu_be_zd, + gen_helper_sve_lddd_be_zd, } } }, - /* Big-endian */ - { { { { gen_helper_sve_ldbds_zsu, - gen_helper_sve_ldhds_be_zsu, - gen_helper_sve_ldsds_be_zsu, - NULL, }, - { gen_helper_sve_ldbdu_zsu, - gen_helper_sve_ldhdu_be_zsu, - gen_helper_sve_ldsdu_be_zsu, - gen_helper_sve_lddd_be_zsu, } }, - { { gen_helper_sve_ldbds_zss, - gen_helper_sve_ldhds_be_zss, - gen_helper_sve_ldsds_be_zss, - NULL, }, - { gen_helper_sve_ldbdu_zss, - gen_helper_sve_ldhdu_be_zss, - gen_helper_sve_ldsdu_be_zss, - gen_helper_sve_lddd_be_zss, } }, - { { gen_helper_sve_ldbds_zd, - gen_helper_sve_ldhds_be_zd, - gen_helper_sve_ldsds_be_zd, - NULL, }, - { gen_helper_sve_ldbdu_zd, - gen_helper_sve_ldhdu_be_zd, - gen_helper_sve_ldsdu_be_zd, - gen_helper_sve_lddd_be_zd, } } }, + /* First-fault */ + { { { gen_helper_sve_ldffbds_zsu, + gen_helper_sve_ldffhds_be_zsu, + gen_helper_sve_ldffsds_be_zsu, + NULL, }, + { gen_helper_sve_ldffbdu_zsu, + gen_helper_sve_ldffhdu_be_zsu, + gen_helper_sve_ldffsdu_be_zsu, + gen_helper_sve_ldffdd_be_zsu, } }, + { { gen_helper_sve_ldffbds_zss, + gen_helper_sve_ldffhds_be_zss, + gen_helper_sve_ldffsds_be_zss, + NULL, }, + { gen_helper_sve_ldffbdu_zss, + gen_helper_sve_ldffhdu_be_zss, + gen_helper_sve_ldffsdu_be_zss, + gen_helper_sve_ldffdd_be_zss, } }, + { { gen_helper_sve_ldffbds_zd, + gen_helper_sve_ldffhds_be_zd, + gen_helper_sve_ldffsds_be_zd, + NULL, }, + { gen_helper_sve_ldffbdu_zd, + gen_helper_sve_ldffhdu_be_zd, + gen_helper_sve_ldffsdu_be_zd, + gen_helper_sve_ldffdd_be_zd, } } } } }, + { /* MTE Active */ + { /* Little-endian */ + { { { gen_helper_sve_ldbds_zsu_mte, + gen_helper_sve_ldhds_le_zsu_mte, + gen_helper_sve_ldsds_le_zsu_mte, + NULL, }, + { gen_helper_sve_ldbdu_zsu_mte, + gen_helper_sve_ldhdu_le_zsu_mte, + gen_helper_sve_ldsdu_le_zsu_mte, + gen_helper_sve_lddd_le_zsu_mte, } }, + { { gen_helper_sve_ldbds_zss_mte, + gen_helper_sve_ldhds_le_zss_mte, + gen_helper_sve_ldsds_le_zss_mte, + NULL, }, + { gen_helper_sve_ldbdu_zss_mte, + gen_helper_sve_ldhdu_le_zss_mte, + gen_helper_sve_ldsdu_le_zss_mte, + gen_helper_sve_lddd_le_zss_mte, } }, + { { gen_helper_sve_ldbds_zd_mte, + gen_helper_sve_ldhds_le_zd_mte, + gen_helper_sve_ldsds_le_zd_mte, + NULL, }, + { gen_helper_sve_ldbdu_zd_mte, + gen_helper_sve_ldhdu_le_zd_mte, + gen_helper_sve_ldsdu_le_zd_mte, + gen_helper_sve_lddd_le_zd_mte, } } }, - /* First-fault */ - { { { gen_helper_sve_ldffbds_zsu, - gen_helper_sve_ldffhds_be_zsu, - gen_helper_sve_ldffsds_be_zsu, - NULL, }, - { gen_helper_sve_ldffbdu_zsu, - gen_helper_sve_ldffhdu_be_zsu, - gen_helper_sve_ldffsdu_be_zsu, - gen_helper_sve_ldffdd_be_zsu, } }, - { { gen_helper_sve_ldffbds_zss, - gen_helper_sve_ldffhds_be_zss, - gen_helper_sve_ldffsds_be_zss, - NULL, }, - { gen_helper_sve_ldffbdu_zss, - gen_helper_sve_ldffhdu_be_zss, - gen_helper_sve_ldffsdu_be_zss, - gen_helper_sve_ldffdd_be_zss, } }, - { { gen_helper_sve_ldffbds_zd, - gen_helper_sve_ldffhds_be_zd, - gen_helper_sve_ldffsds_be_zd, - NULL, }, - { gen_helper_sve_ldffbdu_zd, - gen_helper_sve_ldffhdu_be_zd, - gen_helper_sve_ldffsdu_be_zd, - gen_helper_sve_ldffdd_be_zd, } } } }, + /* First-fault */ + { { { gen_helper_sve_ldffbds_zsu_mte, + gen_helper_sve_ldffhds_le_zsu_mte, + gen_helper_sve_ldffsds_le_zsu_mte, + NULL, }, + { gen_helper_sve_ldffbdu_zsu_mte, + gen_helper_sve_ldffhdu_le_zsu_mte, + gen_helper_sve_ldffsdu_le_zsu_mte, + gen_helper_sve_ldffdd_le_zsu_mte, } }, + { { gen_helper_sve_ldffbds_zss_mte, + gen_helper_sve_ldffhds_le_zss_mte, + gen_helper_sve_ldffsds_le_zss_mte, + NULL, }, + { gen_helper_sve_ldffbdu_zss_mte, + gen_helper_sve_ldffhdu_le_zss_mte, + gen_helper_sve_ldffsdu_le_zss_mte, + gen_helper_sve_ldffdd_le_zss_mte, } }, + { { gen_helper_sve_ldffbds_zd_mte, + gen_helper_sve_ldffhds_le_zd_mte, + gen_helper_sve_ldffsds_le_zd_mte, + NULL, }, + { gen_helper_sve_ldffbdu_zd_mte, + gen_helper_sve_ldffhdu_le_zd_mte, + gen_helper_sve_ldffsdu_le_zd_mte, + gen_helper_sve_ldffdd_le_zd_mte, } } } }, + { /* Big-endian */ + { { { gen_helper_sve_ldbds_zsu_mte, + gen_helper_sve_ldhds_be_zsu_mte, + gen_helper_sve_ldsds_be_zsu_mte, + NULL, }, + { gen_helper_sve_ldbdu_zsu_mte, + gen_helper_sve_ldhdu_be_zsu_mte, + gen_helper_sve_ldsdu_be_zsu_mte, + gen_helper_sve_lddd_be_zsu_mte, } }, + { { gen_helper_sve_ldbds_zss_mte, + gen_helper_sve_ldhds_be_zss_mte, + gen_helper_sve_ldsds_be_zss_mte, + NULL, }, + { gen_helper_sve_ldbdu_zss_mte, + gen_helper_sve_ldhdu_be_zss_mte, + gen_helper_sve_ldsdu_be_zss_mte, + gen_helper_sve_lddd_be_zss_mte, } }, + { { gen_helper_sve_ldbds_zd_mte, + gen_helper_sve_ldhds_be_zd_mte, + gen_helper_sve_ldsds_be_zd_mte, + NULL, }, + { gen_helper_sve_ldbdu_zd_mte, + gen_helper_sve_ldhdu_be_zd_mte, + gen_helper_sve_ldsdu_be_zd_mte, + gen_helper_sve_lddd_be_zd_mte, } } }, + + /* First-fault */ + { { { gen_helper_sve_ldffbds_zsu_mte, + gen_helper_sve_ldffhds_be_zsu_mte, + gen_helper_sve_ldffsds_be_zsu_mte, + NULL, }, + { gen_helper_sve_ldffbdu_zsu_mte, + gen_helper_sve_ldffhdu_be_zsu_mte, + gen_helper_sve_ldffsdu_be_zsu_mte, + gen_helper_sve_ldffdd_be_zsu_mte, } }, + { { gen_helper_sve_ldffbds_zss_mte, + gen_helper_sve_ldffhds_be_zss_mte, + gen_helper_sve_ldffsds_be_zss_mte, + NULL, }, + { gen_helper_sve_ldffbdu_zss_mte, + gen_helper_sve_ldffhdu_be_zss_mte, + gen_helper_sve_ldffsdu_be_zss_mte, + gen_helper_sve_ldffdd_be_zss_mte, } }, + { { gen_helper_sve_ldffbds_zd_mte, + gen_helper_sve_ldffhds_be_zd_mte, + gen_helper_sve_ldffsds_be_zd_mte, + NULL, }, + { gen_helper_sve_ldffbdu_zd_mte, + gen_helper_sve_ldffhdu_be_zd_mte, + gen_helper_sve_ldffsdu_be_zd_mte, + gen_helper_sve_ldffdd_be_zd_mte, } } } } }, }; static bool trans_LD1_zprz(DisasContext *s, arg_LD1_zprz *a) { gen_helper_gvec_mem_scatter *fn = NULL; - int be = s->be_data == MO_BE; + bool be = s->be_data == MO_BE; + bool mte = s->mte_active[0]; if (!sve_access_check(s)) { return true; @@ -5407,23 +5578,24 @@ static bool trans_LD1_zprz(DisasContext *s, arg_LD1_zprz *a) switch (a->esz) { case MO_32: - fn = gather_load_fn32[be][a->ff][a->xs][a->u][a->msz]; + fn = gather_load_fn32[mte][be][a->ff][a->xs][a->u][a->msz]; break; case MO_64: - fn = gather_load_fn64[be][a->ff][a->xs][a->u][a->msz]; + fn = gather_load_fn64[mte][be][a->ff][a->xs][a->u][a->msz]; break; } assert(fn != NULL); do_mem_zpz(s, a->rd, a->pg, a->rm, a->scale * a->msz, - cpu_reg_sp(s, a->rn), a->msz, fn); + cpu_reg_sp(s, a->rn), a->msz, false, fn); return true; } static bool trans_LD1_zpiz(DisasContext *s, arg_LD1_zpiz *a) { gen_helper_gvec_mem_scatter *fn = NULL; - int be = s->be_data == MO_BE; + bool be = s->be_data == MO_BE; + bool mte = s->mte_active[0]; TCGv_i64 imm; if (a->esz < a->msz || (a->esz == a->msz && !a->u)) { @@ -5435,10 +5607,10 @@ static bool trans_LD1_zpiz(DisasContext *s, arg_LD1_zpiz *a) switch (a->esz) { case MO_32: - fn = gather_load_fn32[be][a->ff][0][a->u][a->msz]; + fn = gather_load_fn32[mte][be][a->ff][0][a->u][a->msz]; break; case MO_64: - fn = gather_load_fn64[be][a->ff][2][a->u][a->msz]; + fn = gather_load_fn64[mte][be][a->ff][2][a->u][a->msz]; break; } assert(fn != NULL); @@ -5447,63 +5619,108 @@ static bool trans_LD1_zpiz(DisasContext *s, arg_LD1_zpiz *a) * by loading the immediate into the scalar parameter. */ imm = tcg_const_i64(a->imm << a->msz); - do_mem_zpz(s, a->rd, a->pg, a->rn, 0, imm, a->msz, fn); + do_mem_zpz(s, a->rd, a->pg, a->rn, 0, imm, a->msz, false, fn); tcg_temp_free_i64(imm); return true; } -/* Indexed by [be][xs][msz]. */ -static gen_helper_gvec_mem_scatter * const scatter_store_fn32[2][2][3] = { - /* Little-endian */ - { { gen_helper_sve_stbs_zsu, - gen_helper_sve_sths_le_zsu, - gen_helper_sve_stss_le_zsu, }, - { gen_helper_sve_stbs_zss, - gen_helper_sve_sths_le_zss, - gen_helper_sve_stss_le_zss, } }, - /* Big-endian */ - { { gen_helper_sve_stbs_zsu, - gen_helper_sve_sths_be_zsu, - gen_helper_sve_stss_be_zsu, }, - { gen_helper_sve_stbs_zss, - gen_helper_sve_sths_be_zss, - gen_helper_sve_stss_be_zss, } }, +/* Indexed by [mte][be][xs][msz]. */ +static gen_helper_gvec_mem_scatter * const scatter_store_fn32[2][2][2][3] = { + { /* MTE Inactive */ + { /* Little-endian */ + { gen_helper_sve_stbs_zsu, + gen_helper_sve_sths_le_zsu, + gen_helper_sve_stss_le_zsu, }, + { gen_helper_sve_stbs_zss, + gen_helper_sve_sths_le_zss, + gen_helper_sve_stss_le_zss, } }, + { /* Big-endian */ + { gen_helper_sve_stbs_zsu, + gen_helper_sve_sths_be_zsu, + gen_helper_sve_stss_be_zsu, }, + { gen_helper_sve_stbs_zss, + gen_helper_sve_sths_be_zss, + gen_helper_sve_stss_be_zss, } } }, + { /* MTE Active */ + { /* Little-endian */ + { gen_helper_sve_stbs_zsu_mte, + gen_helper_sve_sths_le_zsu_mte, + gen_helper_sve_stss_le_zsu_mte, }, + { gen_helper_sve_stbs_zss_mte, + gen_helper_sve_sths_le_zss_mte, + gen_helper_sve_stss_le_zss_mte, } }, + { /* Big-endian */ + { gen_helper_sve_stbs_zsu_mte, + gen_helper_sve_sths_be_zsu_mte, + gen_helper_sve_stss_be_zsu_mte, }, + { gen_helper_sve_stbs_zss_mte, + gen_helper_sve_sths_be_zss_mte, + gen_helper_sve_stss_be_zss_mte, } } }, }; /* Note that we overload xs=2 to indicate 64-bit offset. */ -static gen_helper_gvec_mem_scatter * const scatter_store_fn64[2][3][4] = { - /* Little-endian */ - { { gen_helper_sve_stbd_zsu, - gen_helper_sve_sthd_le_zsu, - gen_helper_sve_stsd_le_zsu, - gen_helper_sve_stdd_le_zsu, }, - { gen_helper_sve_stbd_zss, - gen_helper_sve_sthd_le_zss, - gen_helper_sve_stsd_le_zss, - gen_helper_sve_stdd_le_zss, }, - { gen_helper_sve_stbd_zd, - gen_helper_sve_sthd_le_zd, - gen_helper_sve_stsd_le_zd, - gen_helper_sve_stdd_le_zd, } }, - /* Big-endian */ - { { gen_helper_sve_stbd_zsu, - gen_helper_sve_sthd_be_zsu, - gen_helper_sve_stsd_be_zsu, - gen_helper_sve_stdd_be_zsu, }, - { gen_helper_sve_stbd_zss, - gen_helper_sve_sthd_be_zss, - gen_helper_sve_stsd_be_zss, - gen_helper_sve_stdd_be_zss, }, - { gen_helper_sve_stbd_zd, - gen_helper_sve_sthd_be_zd, - gen_helper_sve_stsd_be_zd, - gen_helper_sve_stdd_be_zd, } }, +static gen_helper_gvec_mem_scatter * const scatter_store_fn64[2][2][3][4] = { + { /* MTE Inactive */ + { /* Little-endian */ + { gen_helper_sve_stbd_zsu, + gen_helper_sve_sthd_le_zsu, + gen_helper_sve_stsd_le_zsu, + gen_helper_sve_stdd_le_zsu, }, + { gen_helper_sve_stbd_zss, + gen_helper_sve_sthd_le_zss, + gen_helper_sve_stsd_le_zss, + gen_helper_sve_stdd_le_zss, }, + { gen_helper_sve_stbd_zd, + gen_helper_sve_sthd_le_zd, + gen_helper_sve_stsd_le_zd, + gen_helper_sve_stdd_le_zd, } }, + { /* Big-endian */ + { gen_helper_sve_stbd_zsu, + gen_helper_sve_sthd_be_zsu, + gen_helper_sve_stsd_be_zsu, + gen_helper_sve_stdd_be_zsu, }, + { gen_helper_sve_stbd_zss, + gen_helper_sve_sthd_be_zss, + gen_helper_sve_stsd_be_zss, + gen_helper_sve_stdd_be_zss, }, + { gen_helper_sve_stbd_zd, + gen_helper_sve_sthd_be_zd, + gen_helper_sve_stsd_be_zd, + gen_helper_sve_stdd_be_zd, } } }, + { /* MTE Inactive */ + { /* Little-endian */ + { gen_helper_sve_stbd_zsu_mte, + gen_helper_sve_sthd_le_zsu_mte, + gen_helper_sve_stsd_le_zsu_mte, + gen_helper_sve_stdd_le_zsu_mte, }, + { gen_helper_sve_stbd_zss_mte, + gen_helper_sve_sthd_le_zss_mte, + gen_helper_sve_stsd_le_zss_mte, + gen_helper_sve_stdd_le_zss_mte, }, + { gen_helper_sve_stbd_zd_mte, + gen_helper_sve_sthd_le_zd_mte, + gen_helper_sve_stsd_le_zd_mte, + gen_helper_sve_stdd_le_zd_mte, } }, + { /* Big-endian */ + { gen_helper_sve_stbd_zsu_mte, + gen_helper_sve_sthd_be_zsu_mte, + gen_helper_sve_stsd_be_zsu_mte, + gen_helper_sve_stdd_be_zsu_mte, }, + { gen_helper_sve_stbd_zss_mte, + gen_helper_sve_sthd_be_zss_mte, + gen_helper_sve_stsd_be_zss_mte, + gen_helper_sve_stdd_be_zss_mte, }, + { gen_helper_sve_stbd_zd_mte, + gen_helper_sve_sthd_be_zd_mte, + gen_helper_sve_stsd_be_zd_mte, + gen_helper_sve_stdd_be_zd_mte, } } }, }; static bool trans_ST1_zprz(DisasContext *s, arg_ST1_zprz *a) { gen_helper_gvec_mem_scatter *fn; - int be = s->be_data == MO_BE; + bool be = s->be_data == MO_BE; + bool mte = s->mte_active[0]; if (a->esz < a->msz || (a->msz == 0 && a->scale)) { return false; @@ -5513,23 +5730,24 @@ static bool trans_ST1_zprz(DisasContext *s, arg_ST1_zprz *a) } switch (a->esz) { case MO_32: - fn = scatter_store_fn32[be][a->xs][a->msz]; + fn = scatter_store_fn32[mte][be][a->xs][a->msz]; break; case MO_64: - fn = scatter_store_fn64[be][a->xs][a->msz]; + fn = scatter_store_fn64[mte][be][a->xs][a->msz]; break; default: g_assert_not_reached(); } do_mem_zpz(s, a->rd, a->pg, a->rm, a->scale * a->msz, - cpu_reg_sp(s, a->rn), a->msz, fn); + cpu_reg_sp(s, a->rn), a->msz, true, fn); return true; } static bool trans_ST1_zpiz(DisasContext *s, arg_ST1_zpiz *a) { gen_helper_gvec_mem_scatter *fn = NULL; - int be = s->be_data == MO_BE; + bool be = s->be_data == MO_BE; + bool mte = s->mte_active[0]; TCGv_i64 imm; if (a->esz < a->msz) { @@ -5541,10 +5759,10 @@ static bool trans_ST1_zpiz(DisasContext *s, arg_ST1_zpiz *a) switch (a->esz) { case MO_32: - fn = scatter_store_fn32[be][0][a->msz]; + fn = scatter_store_fn32[mte][be][0][a->msz]; break; case MO_64: - fn = scatter_store_fn64[be][2][a->msz]; + fn = scatter_store_fn64[mte][be][2][a->msz]; break; } assert(fn != NULL); @@ -5553,7 +5771,7 @@ static bool trans_ST1_zpiz(DisasContext *s, arg_ST1_zpiz *a) * by loading the immediate into the scalar parameter. */ imm = tcg_const_i64(a->imm << a->msz); - do_mem_zpz(s, a->rd, a->pg, a->rn, 0, imm, a->msz, fn); + do_mem_zpz(s, a->rd, a->pg, a->rn, 0, imm, a->msz, true, fn); tcg_temp_free_i64(imm); return true; } From patchwork Wed Jun 3 01:13:11 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 187435 Delivered-To: patch@linaro.org Received: by 2002:a92:cf06:0:0:0:0:0 with SMTP id c6csp79405ilo; Tue, 2 Jun 2020 18:30:25 -0700 (PDT) X-Google-Smtp-Source: ABdhPJw3hT0uR87pMrqjzDm4Ngva47Tq2/m3ixBD8RNlcKgrgpT5pyHYPjsnY3wzVNBwwsLPTAxv X-Received: by 2002:a25:db06:: with SMTP id g6mr43011512ybf.401.1591147825514; Tue, 02 Jun 2020 18:30:25 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1591147825; cv=none; d=google.com; s=arc-20160816; b=QXAjR1vjwfLSkRk8D8/G/h2DjuVmofu9q5/erjZK7Ygg8GGHiRsrJ2X9ri4JCMknjX g1yGSZ21z6FdAqVqrVQ3GYgqEDcbzbQLOcjqvOyBMB695tzxGJcepQJs3BOfvCAbsQwY IXbvMy9BWvD151nUETcypFfNl40g6fKyYQILyz2PEEA6eT9FjWHFWWknG9lbvGpPhvdF QsBbXG0vL2mAH1k/4pnKxruK+1FnV/Qi1XM3sqC6MRxgEJNLz9nKB3BCZR9M4ZbywrJN MEsk1WKjEjZ7roCyLnfXPXnR0JNCueG3XDcL/KhgrxKN73e20eCvmqKV3Uu9KSJcoYyC 2Hbg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=L5HmehI0u6Wl5CalJmhYTRnudOl8IyN54sr+tdgMFy8=; b=jIq/3wWMxFEHdZt4rBVZ0FBYM9G39jmEgpz0sGmGz2DJpphJOgghMv37ISANKgWmj2 wRmUwe4TqghjH0xQZju5yDFjchi3BA4F37I38C419Jz7mLNRsAqtqAE6y8uY9OZDr1yJ jjbiutFLRtNQ0YzhxMYt16E693V6h8vVQehxyCtXR9rKS9nVMS2NRhTLmtYPeTG4E1Pe MC1H2yfMvNXcTjuqeXeuGxp9mx4q634y316KX5WIVlilP4aSZfk5tYZ1WRCBJEleiyDd YV5VSuHtG7XelCw2YnzIp4/LETkSRi0zF6Ik1rG4VD6BvXDdg1kmbHR5IxZLQkm9ZUsl 913g== ARC-Authentication-Results: i=1; mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=GQ5fwHhf; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id y78si418105ybe.61.2020.06.02.18.30.25 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 02 Jun 2020 18:30:25 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=GQ5fwHhf; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:38982 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jgIEa-0001u2-Qa for patch@linaro.org; Tue, 02 Jun 2020 21:30:24 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:49018) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jgHyq-00052e-Kl for qemu-devel@nongnu.org; Tue, 02 Jun 2020 21:14:08 -0400 Received: from mail-pf1-x430.google.com ([2607:f8b0:4864:20::430]:40926) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1jgHyp-0003gd-Ob for qemu-devel@nongnu.org; Tue, 02 Jun 2020 21:14:08 -0400 Received: by mail-pf1-x430.google.com with SMTP id v2so415519pfv.7 for ; Tue, 02 Jun 2020 18:14:07 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=L5HmehI0u6Wl5CalJmhYTRnudOl8IyN54sr+tdgMFy8=; b=GQ5fwHhfRmrfSQOiJOp+xitMa1bz0gG0vGhe8roDv+urgOknORAf0CBlXKE2xDBN/2 ctgM53TJTuO+57u6u/KnPugD0jox3VSKKY798vuYy+GKSQz9lAM+d13vEba0ce1MXRaS L3mDKJezZ7ELO3jMVUC6jSU31XSO9edrSFAEzvwFczmAh1dRkCipTzPlnewuUbIN7rom 30RTmE3+U3198xpc5f+PSWwZrGltaJcBXvUEkFvwmOQRK1Dc2rqZB/s9NOM+zr+XwNNa x0CX/vcqmfs/Yj9IO4Md/AOjWWZ9vg1Z5Baa7gPxtbIVmnLh7HzmPcMsnGr8uIg2KzYn +51Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=L5HmehI0u6Wl5CalJmhYTRnudOl8IyN54sr+tdgMFy8=; b=GOxZ/6rAmb4M2GEw65aRPFY0ItOyvgoQtYfJ4mCuBdDvb8+F3yrAkpV5dcFsNIw2ou s0BUQJM4r8etzXeu75lgDYEu161dSfh2PyGDPCy/1Wpea/7Wvj5OvZF8VRPRB4vb/pvF CoPKLP/ttfu0hg+RhSMKsFekKHT+LfKlt+QWXf9JK7ShoXV5MIuVx2/ZeRU+t1pn3Mq8 LRwcTrrY0Uqq4Ry5e+N/QDu4c8f0rW/4ZU2ZcZK4zkZBV6kzAuzKa58iaXhyfusO0naV cVQBTlCOaaZtiCmn4dZckk1XSTOtnI6GgelDkqQYmRDpJaBUdw4JW9lu7Kag1N9eKAva tlmw== X-Gm-Message-State: AOAM533+aW23XH6dMZ8vlhiP3TMi512s747d2MYcVOSDUM67mvABmNAv k2wyZi4C8AHnssPQGTF9pHvGNi+IpJ0= X-Received: by 2002:a62:7707:: with SMTP id s7mr28622157pfc.90.1591146845950; Tue, 02 Jun 2020 18:14:05 -0700 (PDT) Received: from localhost.localdomain (174-21-143-238.tukw.qwest.net. [174.21.143.238]) by smtp.gmail.com with ESMTPSA id 3sm290067pfe.85.2020.06.02.18.14.05 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 02 Jun 2020 18:14:05 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v7 36/42] target/arm: Complete TBI clearing for user-only for SVE Date: Tue, 2 Jun 2020 18:13:11 -0700 Message-Id: <20200603011317.473934-37-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200603011317.473934-1-richard.henderson@linaro.org> References: <20200603011317.473934-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::430; envelope-from=richard.henderson@linaro.org; helo=mail-pf1-x430.google.com X-detected-operating-system: by eggs.gnu.org: No matching host in p0f cache. That's all we know. X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: peter.maydell@linaro.org, qemu-arm@nongnu.org, steplong@quicinc.com Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" There are a number of paths by which the TBI is still intact for user-only in the SVE helpers. Because we currently always set TBI for user-only, we do not need to pass down the actual TBI setting from above, and we can remove the top byte in the inner-most primitives, so that none are forgotten. Moreover, this keeps the "dirty" pointer around at the higher levels, where we need it for any MTE checking. Since the normal case, especially for user-only, goes through RAM, this clearing merely adds two insns per page lookup, which will be completely in the noise. Signed-off-by: Richard Henderson --- target/arm/sve_helper.c | 19 +++++++++++++++++-- 1 file changed, 17 insertions(+), 2 deletions(-) -- 2.25.1 diff --git a/target/arm/sve_helper.c b/target/arm/sve_helper.c index 2d2aa14b43..460a3e5937 100644 --- a/target/arm/sve_helper.c +++ b/target/arm/sve_helper.c @@ -3966,14 +3966,16 @@ static void sve_##NAME##_host(void *vd, intptr_t reg_off, void *host) \ static void sve_##NAME##_tlb(CPUARMState *env, void *vd, intptr_t reg_off, \ target_ulong addr, uintptr_t ra) \ { \ - *(TYPEE *)(vd + H(reg_off)) = (TYPEM)TLB(env, addr, ra); \ + *(TYPEE *)(vd + H(reg_off)) = \ + (TYPEM)TLB(env, useronly_clean_ptr(addr), ra); \ } #define DO_ST_TLB(NAME, H, TYPEE, TYPEM, TLB) \ static void sve_##NAME##_tlb(CPUARMState *env, void *vd, intptr_t reg_off, \ target_ulong addr, uintptr_t ra) \ { \ - TLB(env, addr, (TYPEM)*(TYPEE *)(vd + H(reg_off)), ra); \ + TLB(env, useronly_clean_ptr(addr), \ + (TYPEM)*(TYPEE *)(vd + H(reg_off)), ra); \ } #define DO_LD_PRIM_1(NAME, H, TE, TM) \ @@ -4091,6 +4093,19 @@ static bool sve_probe_page(SVEHostPage *info, bool nofault, int flags; addr += mem_off; + + /* + * User-only currently always issues with TBI. See the comment + * above useronly_clean_ptr. Usually we clean this top byte away + * during translation, but we can't do that for e.g. vector + imm + * addressing modes. + * + * We currently always enable TBI for user-only, and do not provide + * a way to turn it off. So clean the pointer unconditionally here, + * rather than look it up here, or pass it down from above. + */ + addr = useronly_clean_ptr(addr); + flags = probe_access_flags(env, addr, access_type, mmu_idx, nofault, &info->host, retaddr); info->flags = flags; From patchwork Wed Jun 3 01:13:12 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 187440 Delivered-To: patch@linaro.org Received: by 2002:a92:cf06:0:0:0:0:0 with SMTP id c6csp81371ilo; Tue, 2 Jun 2020 18:33:18 -0700 (PDT) X-Google-Smtp-Source: ABdhPJz2GxSE4jhQHjRc5Iuup3fpead4Pr46k40r4dAfQTsQ98yQNDPNVXH+OJq0UYr8YcDN66NF X-Received: by 2002:a5b:3c9:: with SMTP id t9mr50709766ybp.40.1591147998837; Tue, 02 Jun 2020 18:33:18 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1591147998; cv=none; d=google.com; s=arc-20160816; b=RDjY1sx119WS1bZMzTocQf099EHluY8biqRizoHjoD7lAhVNKYh23s39xJucz15gkP KJuqqxD9sD/Tgyu20i5a3y67XUZ9NAc1BVEedLgzvq0whS9jy934GoEHbX/j014767zp dTK2+XUzzwK4XVdAMG05Y2caR/gXSZhUHv1oibrShKMTVrao2YJqfWGHxuxXY2vXpZER wJEE2oPWlGooJsFi8llzk7edUXcONSD/v4mn3C9GNIUeHl6IgzwOwA3/V4+foda2dUyO TBIRbLB7sL5XAmMKstyGROFdRxBa9EOuWf2z25LiBGJoo8aIFzO9Pm43wYg/H8cqUxYh BHTw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=CSv1KeR8menjHwPFPDbJzuFwzIdMp1D6EtoFs3JeUrg=; b=teGM5e/w4e729bVEzYs15sr+u8BRdX41wUeUxBRr68N9P3nVjUrsVDwpfAMogSbDxT 7QDSkIJO2b4N2sE9uyj48kqdeOEE48Ej0CLRdmXod64IeJpoPo76ZVRY+j943vBSVMfG ZeEHLWCCGz0cnrrnIi2HxggpRWazIcvg76J4HMyzJVuNrHM7D1XjiMX/UjQq2HgLGcbl 1tIA0yrNVtCvg483V/Uw6mECWHpBxMnxXBYUjjhDX1avnIBEUJ5U+l/eAt+mlOXPBoRO 17bsNsRRDtGPnitm2vIahvbvJfa+z0xziYoHYybg/rFYwmSvQD1IOiEVNg6ftHX6I7Hn kjrw== ARC-Authentication-Results: i=1; mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b="cTI/2G4x"; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id l4si460661ybt.230.2020.06.02.18.33.18 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 02 Jun 2020 18:33:18 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b="cTI/2G4x"; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:53570 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jgIHO-0007oC-9a for patch@linaro.org; Tue, 02 Jun 2020 21:33:18 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:49054) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jgHyt-0005BZ-Ns for qemu-devel@nongnu.org; Tue, 02 Jun 2020 21:14:11 -0400 Received: from mail-pg1-x542.google.com ([2607:f8b0:4864:20::542]:35919) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1jgHyr-0003h8-FS for qemu-devel@nongnu.org; Tue, 02 Jun 2020 21:14:11 -0400 Received: by mail-pg1-x542.google.com with SMTP id t7so572993pgt.3 for ; Tue, 02 Jun 2020 18:14:09 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=CSv1KeR8menjHwPFPDbJzuFwzIdMp1D6EtoFs3JeUrg=; b=cTI/2G4xtLZWFI74ebrmfOp+95jUiunM6RjKXxwCHPVrtNVaCIBNTMhtpACOMNWsUA En42YXxQrHeCj2YRftrnCowP8VAY47gnwc8IEnEPZ15BVQBWbiZvjpUAP4nX8tbfHc2N gMLxjGY801ATYOsNPHzIT8pmXGrNSFoM94nAM1HE+r5OqTWUMCWV6LXUY4wikWtMPiat NRi+8AU1f8lGeZJMppvNWGQRgi5e6UNo820ki8gu7yhSMStaYwxeBqxH8NJ/VxdCdsYr w3WjeGNsV31OzeQ9QDKunb5Y7dvqZVQUUAbn+uV4885fngZcIoY+NuRPZ0PwG8pwzHvT J2TQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=CSv1KeR8menjHwPFPDbJzuFwzIdMp1D6EtoFs3JeUrg=; b=AYfATEeFsI1yFKzrGw1JTvu+WfAbPpIqdm6Y6vVdK4Oy8mG7np43aGwj20iqQrV8Ue jxCGBgzHOSbDSaMChWd9M/eL1cM763bz3qqN9W6wrL+McmkXK7ngy/y2xrz0ppFZMHtQ xbDpdNdCEURRbyJRqLwg48USTXWbBqr2xRCMWP/6BYr8U/oZNYZb+DhLg2k5VeGQ+8xb ckTnEFWfzvIghmuVznchBoLVu+G+QpfObwqFOu3q618a0gpnF5QFgzURkYG6rTq/3eBl xMKBLScjc7+o6s5stnExsPLKbpG10G2nUyH4wfoCyI+DjdIS+MBHbkS2PRxuZ0i5vaAT 3STw== X-Gm-Message-State: AOAM533C/S2GiaaA6UPpAN/D0kMYp8aBM8fVVtYfWpOCd4N7GT85wvXB Y/R6f0d53857SuIqTz3ExJObUAMAc4Q= X-Received: by 2002:aa7:868f:: with SMTP id d15mr6072371pfo.166.1591146847684; Tue, 02 Jun 2020 18:14:07 -0700 (PDT) Received: from localhost.localdomain (174-21-143-238.tukw.qwest.net. [174.21.143.238]) by smtp.gmail.com with ESMTPSA id 3sm290067pfe.85.2020.06.02.18.14.06 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 02 Jun 2020 18:14:06 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v7 37/42] target/arm: Implement data cache set allocation tags Date: Tue, 2 Jun 2020 18:13:12 -0700 Message-Id: <20200603011317.473934-38-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200603011317.473934-1-richard.henderson@linaro.org> References: <20200603011317.473934-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::542; envelope-from=richard.henderson@linaro.org; helo=mail-pg1-x542.google.com X-detected-operating-system: by eggs.gnu.org: No matching host in p0f cache. That's all we know. X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: peter.maydell@linaro.org, qemu-arm@nongnu.org, steplong@quicinc.com Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" This is DC GVA and DC GZVA, and the tag check for DC ZVA. Signed-off-by: Richard Henderson --- v2: Use allocation_tag_mem + memset. v3: Require pre-cleaned addresses. v6: Move DCZ block size assert to cpu realize. Perform a tag check for DC ZVA. --- target/arm/cpu.h | 4 +++- target/arm/helper.c | 16 ++++++++++++++++ target/arm/translate-a64.c | 39 ++++++++++++++++++++++++++++++++++++++ 3 files changed, 58 insertions(+), 1 deletion(-) -- 2.25.1 Reviewed-by: Peter Maydell diff --git a/target/arm/cpu.h b/target/arm/cpu.h index a5d3b6c9ee..17594226eb 100644 --- a/target/arm/cpu.h +++ b/target/arm/cpu.h @@ -2360,7 +2360,9 @@ static inline uint64_t cpreg_to_kvm_id(uint32_t cpregid) #define ARM_CP_NZCV (ARM_CP_SPECIAL | 0x0300) #define ARM_CP_CURRENTEL (ARM_CP_SPECIAL | 0x0400) #define ARM_CP_DC_ZVA (ARM_CP_SPECIAL | 0x0500) -#define ARM_LAST_SPECIAL ARM_CP_DC_ZVA +#define ARM_CP_DC_GVA (ARM_CP_SPECIAL | 0x0600) +#define ARM_CP_DC_GZVA (ARM_CP_SPECIAL | 0x0700) +#define ARM_LAST_SPECIAL ARM_CP_DC_GZVA #define ARM_CP_FPU 0x1000 #define ARM_CP_SVE 0x2000 #define ARM_CP_NO_GDB 0x4000 diff --git a/target/arm/helper.c b/target/arm/helper.c index c56e8e500b..3adafc07f0 100644 --- a/target/arm/helper.c +++ b/target/arm/helper.c @@ -6996,6 +6996,22 @@ static const ARMCPRegInfo mte_el0_cacheop_reginfo[] = { .opc0 = 1, .opc1 = 3, .crn = 7, .crm = 14, .opc2 = 5, .type = ARM_CP_NOP, .access = PL0_W, .accessfn = aa64_cacheop_poc_access }, + { .name = "DC_GVA", .state = ARM_CP_STATE_AA64, + .opc0 = 1, .opc1 = 3, .crn = 7, .crm = 4, .opc2 = 3, + .access = PL0_W, .type = ARM_CP_DC_GVA, +#ifndef CONFIG_USER_ONLY + /* Avoid overhead of an access check that always passes in user-mode */ + .accessfn = aa64_zva_access, +#endif + }, + { .name = "DC_GZVA", .state = ARM_CP_STATE_AA64, + .opc0 = 1, .opc1 = 3, .crn = 7, .crm = 4, .opc2 = 4, + .access = PL0_W, .type = ARM_CP_DC_GZVA, +#ifndef CONFIG_USER_ONLY + /* Avoid overhead of an access check that always passes in user-mode */ + .accessfn = aa64_zva_access, +#endif + }, REGINFO_SENTINEL }; diff --git a/target/arm/translate-a64.c b/target/arm/translate-a64.c index 394633babd..009cea6ed1 100644 --- a/target/arm/translate-a64.c +++ b/target/arm/translate-a64.c @@ -1865,6 +1865,45 @@ static void handle_sys(DisasContext *s, uint32_t insn, bool isread, } gen_helper_dc_zva(cpu_env, tcg_rt); return; + case ARM_CP_DC_GVA: + { + TCGv_i64 clean_addr, tag; + + /* + * DC_GVA, like DC_ZVA, requires that we supply the original + * pointer for an invalid page. Probe that address first. + */ + tcg_rt = cpu_reg(s, rt); + clean_addr = clean_data_tbi(s, tcg_rt); + gen_probe_access(s, clean_addr, MMU_DATA_STORE, MO_8); + + if (s->ata) { + /* Extract the tag from the register to match STZGM. */ + tag = tcg_temp_new_i64(); + tcg_gen_shri_i64(tag, tcg_rt, 56); + gen_helper_stzgm_tags(cpu_env, clean_addr, tag); + tcg_temp_free_i64(tag); + } + } + return; + case ARM_CP_DC_GZVA: + { + TCGv_i64 clean_addr, tag; + + /* For DC_GZVA, we can rely on DC_ZVA for the proper fault. */ + tcg_rt = cpu_reg(s, rt); + clean_addr = clean_data_tbi(s, tcg_rt); + gen_helper_dc_zva(cpu_env, clean_addr); + + if (s->ata) { + /* Extract the tag from the register to match STZGM. */ + tag = tcg_temp_new_i64(); + tcg_gen_shri_i64(tag, tcg_rt, 56); + gen_helper_stzgm_tags(cpu_env, clean_addr, tag); + tcg_temp_free_i64(tag); + } + } + return; default: break; } From patchwork Wed Jun 3 01:13:13 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 187442 Delivered-To: patch@linaro.org Received: by 2002:a92:cf06:0:0:0:0:0 with SMTP id c6csp82601ilo; Tue, 2 Jun 2020 18:35:27 -0700 (PDT) X-Google-Smtp-Source: ABdhPJy71II63vhWOon1yhGfY3fXnlFfG/pmd0iabc+WemRmdOStAgifRJKm3nlOK/UcyWhn0a4k X-Received: by 2002:a25:8509:: with SMTP id w9mr44569486ybk.500.1591148127106; Tue, 02 Jun 2020 18:35:27 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1591148127; cv=none; d=google.com; s=arc-20160816; b=C4MLZxJUBGxQco0X+Ae7YL5tYRkOudx98+YjtZy98WHmm51NPQb5f516BH4Q2lcfqX 1tbr5bq+DePf5tEKWOESYDmwwAvyEP5jtmhcm/3RheVSr+PUZ+uCp/VeyqW+smbFlgBX ejZC3sEgduye14L6gL4oSxDca0RYS0P/jgNsFCCMeWte0raxbEUVBmB4RzCx2Iyztkg7 aJffQ75yo/2NJqxNMFAQm17Su+z1iAArPEeWdWptJGqLmkmg7JAq4fBvu7ytzXMAvXY+ XU5+05cfU+n6kYqq8zI99NHZDzw900ZFeEVtEcfos9LHLRcs9da+KLrFLf1fcchYPuUi 6hag== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=uaH1sngv9xlWH/X8rZjic7q7Jho9e1jf1ezwAdUAR9c=; b=DO6h37nCAFmp9d8x93CmPWznmbpovwBkpP3crBtxi2vFNLlPb/C9rpuSjLyftxUqNe 7SrYaYvt++pdsvkeddFl7PY04Mc5/fTW6rNPJzxIFN/OMsx/7YK3cck8rexftD/j6X7Q aDHzKlaSRspLnKBrg7NxSYW9GbZyryjCBLZZleHXYLZd54JBX9PBoyK1vIWh3a52wX6a 4yV14qmlTqRuzBiVQfJZRgFCShpwgHSKStkHTvWuVwjreZTsLPj6br2AohTu9MgJvd2j T7CHqPbQjMs0z8bmjF57fU923pZ2mL7BsXywCKkF1Cbxlmsk6L8ObwvRCFC6P70z/kFo N65g== ARC-Authentication-Results: i=1; mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=W2VpHdhl; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id r1si305233ybk.482.2020.06.02.18.35.27 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 02 Jun 2020 18:35:27 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=W2VpHdhl; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:35454 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jgIJS-0003V9-Lc for patch@linaro.org; Tue, 02 Jun 2020 21:35:26 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:49048) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jgHyt-0005AP-Bd for qemu-devel@nongnu.org; Tue, 02 Jun 2020 21:14:11 -0400 Received: from mail-pl1-x644.google.com ([2607:f8b0:4864:20::644]:39719) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1jgHys-0003hK-JE for qemu-devel@nongnu.org; Tue, 02 Jun 2020 21:14:10 -0400 Received: by mail-pl1-x644.google.com with SMTP id v24so199532plo.6 for ; Tue, 02 Jun 2020 18:14:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=uaH1sngv9xlWH/X8rZjic7q7Jho9e1jf1ezwAdUAR9c=; b=W2VpHdhlmXjJWldCJJ/AWBtSxY3UTYaz44DqxLhKEiskonM1uXUpPuN29hCgBFhPvP gA7YjjtvTHhJ8fx92XaeSMXl0EFMTDiukC9LU3Yk9lybiGySoCD1FPyYl6zvjVsQjQ/1 bQ2EaThf9TApdkDsf/53X/YavAdnEWivRfC/g5t8sKJoQ31aoYdEOBXNRTkUlsEOj/HF pEy7asfNnWtpbtpB+r/l0jBmQgdZTeeqoGSkuw3PxtzqKW4l1RH1ulpfwhNN323u4vCQ 18fu0Mjvz++aU6Vdi5E0zAATT6NdUkmOwx78uuxXr/TJyE2Pn5x+g6UZwh7ulqF3O9zU EG3g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=uaH1sngv9xlWH/X8rZjic7q7Jho9e1jf1ezwAdUAR9c=; b=iNPNSEr/nAz5FBq2KlQyYWCeL7FNzZx1c1toZhctfds+lBE7pvQ0QLf8rpeBKbqc4w y885PdDp2PO2Dy7zXb8j/BfuFPsIpqQIybwmm4IJBfHCcu9scbJhOnmrXPFsA6Czmj5T jLnHmfVZuBdZweEUWZdve1lWahpz1+HByT7koJg4u+jBmIbBeWLOA7DADh+ZRooTP3ZM FIRbToJQAJkD/N9aUZb1AL7XR2yCycraAvdcVG8k1VszUs6H+3wunhc00+gd491DT0zW cDOUycMtQ8LXz51jUtKBlLWsgXwj4qWVTD/otBGahvR8ph2uglQzy5t3zcOSuzbYC4Di K9Tg== X-Gm-Message-State: AOAM530swloX/ZJovSUyoEUyU1IGuG88WYwVpW+Tos02HvzVLz+0VfhQ nDk/dImhWe18qn+I+9hx37fVKyD57HE= X-Received: by 2002:a17:902:d711:: with SMTP id w17mr28235658ply.122.1591146848946; Tue, 02 Jun 2020 18:14:08 -0700 (PDT) Received: from localhost.localdomain (174-21-143-238.tukw.qwest.net. [174.21.143.238]) by smtp.gmail.com with ESMTPSA id 3sm290067pfe.85.2020.06.02.18.14.07 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 02 Jun 2020 18:14:08 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v7 38/42] target/arm: Set PSTATE.TCO on exception entry Date: Tue, 2 Jun 2020 18:13:13 -0700 Message-Id: <20200603011317.473934-39-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200603011317.473934-1-richard.henderson@linaro.org> References: <20200603011317.473934-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::644; envelope-from=richard.henderson@linaro.org; helo=mail-pl1-x644.google.com X-detected-operating-system: by eggs.gnu.org: No matching host in p0f cache. That's all we know. X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: peter.maydell@linaro.org, qemu-arm@nongnu.org, steplong@quicinc.com Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" D1.10 specifies that exception handlers begin with tag checks overridden. Reviewed-by: Peter Maydell Signed-off-by: Richard Henderson --- v2: Only set if MTE feature present. --- target/arm/helper.c | 3 +++ 1 file changed, 3 insertions(+) -- 2.25.1 diff --git a/target/arm/helper.c b/target/arm/helper.c index 3adafc07f0..50717afa4a 100644 --- a/target/arm/helper.c +++ b/target/arm/helper.c @@ -9702,6 +9702,9 @@ static void arm_cpu_do_interrupt_aarch64(CPUState *cs) break; } } + if (cpu_isar_feature(aa64_mte, cpu)) { + new_mode |= PSTATE_TCO; + } pstate_write(env, PSTATE_DAIF | new_mode); env->aarch64 = 1; From patchwork Wed Jun 3 01:13:14 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 187433 Delivered-To: patch@linaro.org Received: by 2002:a92:cf06:0:0:0:0:0 with SMTP id c6csp79330ilo; Tue, 2 Jun 2020 18:30:18 -0700 (PDT) X-Google-Smtp-Source: ABdhPJwNP9lwr4QMqXSfmFryg35T4bpUW71cHoL2inWzNpRAt0uMNtzx4hRST+fxT/J7Lzwft1kr X-Received: by 2002:a5b:511:: with SMTP id o17mr44959954ybp.432.1591147818107; Tue, 02 Jun 2020 18:30:18 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1591147818; cv=none; d=google.com; s=arc-20160816; b=Ox4PsZ3BdC5yEnTWvcuxaGeG2X6kpQxja3fAGXIPTEZUi9oBVx7lTDF1FRsd2HC1Tj fhIlFaaNeBcW8mOedb/7aHWY+hnf+vkWnu/8kSsjdi15gQccWRQ68eo9ZlGsMP8QzwtN 1Lz46txiPMD8NgRyGGhi168SIsa6JSSZHdB3OvBHrNgTLHuS8IxqoUwPkcR2IeUQt6MF 4IfVzL6sTtURkrWhJjA83Lz4MKXDE1nDibgj4rQXGRNBnSMm6NgtwCXoY2/cGfA0n+Hc PwtpLPJXzfii7j2aLYSbGLbVQsdtwnJATFzh4nY7KaSmHEMH+gFlslHsLfti4GlU3/Jx iqNg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=DWHyTpGlKBuharmhIu7/0HNfc6hnYtHUc2NhnIgEMGo=; b=wqLNHXBmrr9zOCSkmWRc6k35S2aIA8drccGaIVitC3uNDjC7RaTu20NAGtUwtT1TRq sat6Hb8U99FPjshqt2Cl8XBgA+2xBPupdzpVD7YfSV0K90sfBHgPhaz0kNxK+zESvPqC kxQa0K2gm6tgBbnY/+QaDh8Oci4tOpbi1c3UHwthKGFx0XAyjV+Rz1CZPwmVk/2ALT9L VeFpG/jKLshGTv+dFETfqwmC9xjxAmassMuH2erW5KFhEBwuUm902fTxAiVAYD6i4n01 xAInY7N5FxUCz9a2jsTH0sEj82rtlp256gboIizWhG0zenERJANUbwxV2QHsxLIFp8/h zK3w== ARC-Authentication-Results: i=1; mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=mmOMn8K6; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id r65si377393ybc.379.2020.06.02.18.30.18 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 02 Jun 2020 18:30:18 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=mmOMn8K6; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:41106 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jgIET-0002jv-JT for patch@linaro.org; Tue, 02 Jun 2020 21:30:17 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:49062) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jgHyu-0005Dr-Go for qemu-devel@nongnu.org; Tue, 02 Jun 2020 21:14:12 -0400 Received: from mail-pl1-x644.google.com ([2607:f8b0:4864:20::644]:36974) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1jgHyt-0003hd-QA for qemu-devel@nongnu.org; Tue, 02 Jun 2020 21:14:12 -0400 Received: by mail-pl1-x644.google.com with SMTP id y18so202368plr.4 for ; Tue, 02 Jun 2020 18:14:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=DWHyTpGlKBuharmhIu7/0HNfc6hnYtHUc2NhnIgEMGo=; b=mmOMn8K6m384XUfdOljdjRtVgurc7mmaDh4q5305ZT+DX+9zE7CHgUDWhfxAgu1War oqTa5to9WH5JJAEL4i/7kpkHMLj6SDRxZcBpkyvBkQb3gUGRrVMD0my4Amkit+/nthO+ cqnOTIty9NPcoVZsoYNhBEUdMqwy9ccjOA0GfxYW3+0XY6aZXWd0WI57uBEwfbebLQ9y 4DaUwiu77oIk32TL+3LhYo4lubrGSMZ401oIg3gS+I3IncQ3HNkE0XnfEtq8sa86cY/e QAcnwujJCpxFOvYgg3LAKHzLc60+JVyRdkBGztARLLQiRb5qS8YKQSdOZjcORVxwVa9y Sldw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=DWHyTpGlKBuharmhIu7/0HNfc6hnYtHUc2NhnIgEMGo=; b=JozGNN9lAL8dwqvTya1Qnj5tbNM+8dSLmr2bZejZyKMDuxJQxnNV1hfJATQHxBWEtO VZO9NBoaEdbTtVxdxwTXK3GXDcxL5+7oJs4kaVBLbqhTHhY5PfRYB3NrUIZ4tjyQhG3v j/QwgqySZqAiLBoso7sVKo4ElBvP+8RX5yq2/NR0cDb4d2k3XB7su3UCO2QusjaiqOAU FMdIuUZadBY0FDPQ2tscPKDLJvUTx2gNxj74/GWjC4Mwfay5mp6IViZhLpjv4pAmTSIG pyxm0He1UsIXrX8qx0wCSZWw7DjmbAzVC4kx6LZtJm8CZpiymEXVBJfIUkExJkDZQX1X d4mg== X-Gm-Message-State: AOAM532U0ePJ5VGdxsnmjt1VmDBnBtZXCi7n5DT5ewaTih1WuvoSuhZQ sgKM5tOckoY+ySu96plBy8dnMv5L+YA= X-Received: by 2002:a17:902:6541:: with SMTP id d1mr25523754pln.96.1591146850191; Tue, 02 Jun 2020 18:14:10 -0700 (PDT) Received: from localhost.localdomain (174-21-143-238.tukw.qwest.net. [174.21.143.238]) by smtp.gmail.com with ESMTPSA id 3sm290067pfe.85.2020.06.02.18.14.09 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 02 Jun 2020 18:14:09 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v7 39/42] target/arm: Enable MTE Date: Tue, 2 Jun 2020 18:13:14 -0700 Message-Id: <20200603011317.473934-40-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200603011317.473934-1-richard.henderson@linaro.org> References: <20200603011317.473934-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::644; envelope-from=richard.henderson@linaro.org; helo=mail-pl1-x644.google.com X-detected-operating-system: by eggs.gnu.org: No matching host in p0f cache. That's all we know. X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: peter.maydell@linaro.org, qemu-arm@nongnu.org, steplong@quicinc.com Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" We now implement all of the components of MTE, without actually supporting any tagged memory. All MTE instructions will work, trivially, so we can enable support. Signed-off-by: Richard Henderson --- v6: Delay user-only cpu reset bits to the user-only patch set. --- target/arm/cpu64.c | 1 + 1 file changed, 1 insertion(+) -- 2.25.1 diff --git a/target/arm/cpu64.c b/target/arm/cpu64.c index cbc5c3868f..e4bce55c5f 100644 --- a/target/arm/cpu64.c +++ b/target/arm/cpu64.c @@ -655,6 +655,7 @@ static void aarch64_max_initfn(Object *obj) t = cpu->isar.id_aa64pfr1; t = FIELD_DP64(t, ID_AA64PFR1, BT, 1); + t = FIELD_DP64(t, ID_AA64PFR1, MTE, 2); cpu->isar.id_aa64pfr1 = t; t = cpu->isar.id_aa64mmfr1; From patchwork Wed Jun 3 01:13:15 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 187443 Delivered-To: patch@linaro.org Received: by 2002:a92:cf06:0:0:0:0:0 with SMTP id c6csp83486ilo; Tue, 2 Jun 2020 18:36:54 -0700 (PDT) X-Google-Smtp-Source: ABdhPJx6Pk5pjIZMsD4u02QtTDkNW7psN2gYNTUq9lADAU0GUNUH9cQ9L41FCsIp9nEmgqiXH9Cg X-Received: by 2002:a25:31c5:: with SMTP id x188mr3397611ybx.295.1591148213942; Tue, 02 Jun 2020 18:36:53 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1591148213; cv=none; d=google.com; s=arc-20160816; b=bouOcXhb/RRDH4QYUqsli/zjgrYyZyNsvpwbxXPv5Gviy0/iusc43C2yfVf+mqwdD0 /ZvwGouR9P6oHQ4DLlG8Up5RFy/6Q3ZyZvV3qEW+n2NTCfeI9rbcfLlWawHS12C6YbvA CbHHOzZdj97+wlccE0W1tGFEuhOmS5bo4oxoeombiio9gpq45Kani/AllybU01j/07cL TgYT/XpDQQbzZB8QJZu5ykHWMVmvpoxr8olj6Z/zSb1UtnDXQI5Ss+5UzgAaV2F9j8bS Uu2CoqHRrn0RnarcHNJQePlBueWlCLRVQVQf/iU6bOpEoOtCSQHiLw1l3Hi4SYK1jJZI +hNg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=665wfLrYyWEoos7hXDIalhn/FRb4JzUf+uVTCWnMdBk=; b=I9Av00x+bvOpQ+6IxblCr2o1VmSTT5oqeSQhFPEERGXG7gG4ogPgmBuEJ230CtB5z5 i05borHTyxeyHVfj32dpDu7yl7fZwS7PUIBMuxXa7zxWwfZlszplYaNR/NZMA1cT3I38 chKSABz+fdMNUh666Gt9SD4YVBZSoCIePp2CAH4oKGz5TEXDqYs7o8FuvRN8EnWa+4YJ PbtTfFg7TN4o47JkdqR20m8aHWEIBTLjAAs7wDEb/ne6lrvqj9gQC7fUmb9HiolaFA6g iHogxVR+opXxCKh54ZzDEtyLUTNBRLKzs2nuM4vf7TmIRq8YoEjPssJQTFym54S/kXXh /K4g== ARC-Authentication-Results: i=1; mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b="RFFLM/rP"; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id g126si364731ybg.335.2020.06.02.18.36.53 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 02 Jun 2020 18:36:53 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b="RFFLM/rP"; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:38882 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jgIKr-00053V-Fg for patch@linaro.org; Tue, 02 Jun 2020 21:36:53 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:49074) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jgHyv-0005HZ-Sa for qemu-devel@nongnu.org; Tue, 02 Jun 2020 21:14:13 -0400 Received: from mail-pl1-x644.google.com ([2607:f8b0:4864:20::644]:45838) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1jgHyv-0003hx-13 for qemu-devel@nongnu.org; Tue, 02 Jun 2020 21:14:13 -0400 Received: by mail-pl1-x644.google.com with SMTP id y11so187605plt.12 for ; Tue, 02 Jun 2020 18:14:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=665wfLrYyWEoos7hXDIalhn/FRb4JzUf+uVTCWnMdBk=; b=RFFLM/rPkSHJRM3cLpBsW4J6geKJtqaYvcYaPsKmbGkUSntjNheN2i6CmXaiEZDUZC qt5SLmlnwXT2YesWxKx0duZ5XYB1hco2WwNNUz3/GPyzOHhSLAiZov5fAi5Gs9g+8L+M mc6t6nmYxdr8mvCz05SOKbGo4FJdWqTYamuh4T+f7HWe9o1y97j6xG+MTPAKdNtc1Dyl SUFtnemT3RBwHfGMAAWSvdBmXM/Qtw3pNMM1bKteFL3wJ+j9MUqtUSXXVLXqXr1ObDWQ /QWFbz806gw43sw+ZfK6UVlfOHo8+tk02rwXZNjTSQkbaaTRc6DmyGkvbVo7wQk9pQwm LlMQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=665wfLrYyWEoos7hXDIalhn/FRb4JzUf+uVTCWnMdBk=; b=uAEWLYzTjPk8ayFnPxh4olPrZdFmb2X6aUOxbKNlNTbuJUzqny1Z5UlsPJCp/3Bohd 2H+nR1B0qRSH7MKvOeB048V/Q0idSpibO4n6s3jJgJTK6q/y+tUcrXAMbjuVGLm9FBch osOTpyPbacBAxP9NCnEmGmPmo7P4hgWBhI0x6y+nshU7Q2ttPI6j1lD6twO7UOiMeA3e oDCKerq1qK93CthXagkCBWQgxhtLaORUMGgh5bXMtnSzhY9U127k/5cCTrcLiptyetBg j7/3quWBS5Xo/Hi16p6hUQjB9y+CBucKxotPETEmMQJUBIe54zlbrlEfM31BRblwhDCD coRA== X-Gm-Message-State: AOAM530fGuM1lwoY7ltGWnypYYBgg+m39L3lpzhsIv0yrcE7qMbh8u68 GPI+rfLBlxYsO6qS9ShFO25rPlOhz8I= X-Received: by 2002:a17:902:a412:: with SMTP id p18mr27270100plq.111.1591146851410; Tue, 02 Jun 2020 18:14:11 -0700 (PDT) Received: from localhost.localdomain (174-21-143-238.tukw.qwest.net. [174.21.143.238]) by smtp.gmail.com with ESMTPSA id 3sm290067pfe.85.2020.06.02.18.14.10 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 02 Jun 2020 18:14:10 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v7 40/42] target/arm: Cache the Tagged bit for a page in MemTxAttrs Date: Tue, 2 Jun 2020 18:13:15 -0700 Message-Id: <20200603011317.473934-41-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200603011317.473934-1-richard.henderson@linaro.org> References: <20200603011317.473934-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::644; envelope-from=richard.henderson@linaro.org; helo=mail-pl1-x644.google.com X-detected-operating-system: by eggs.gnu.org: No matching host in p0f cache. That's all we know. X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: peter.maydell@linaro.org, qemu-arm@nongnu.org, steplong@quicinc.com Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" This "bit" is a particular value of the page's MemAttr. Signed-off-by: Richard Henderson --- v6: Test HCR_EL2.{DC,DCT}; test Stage2 attributes. --- target/arm/helper.c | 43 +++++++++++++++++++++++++++++++++++-------- 1 file changed, 35 insertions(+), 8 deletions(-) -- 2.25.1 diff --git a/target/arm/helper.c b/target/arm/helper.c index 50717afa4a..74925010ea 100644 --- a/target/arm/helper.c +++ b/target/arm/helper.c @@ -10875,6 +10875,7 @@ static bool get_phys_addr_lpae(CPUARMState *env, target_ulong address, uint64_t descaddrmask; bool aarch64 = arm_el_is_aa64(env, el); bool guarded = false; + uint8_t memattr; /* TODO: This code does not support shareability levels. */ if (aarch64) { @@ -11099,17 +11100,32 @@ static bool get_phys_addr_lpae(CPUARMState *env, target_ulong address, txattrs->target_tlb_bit0 = true; } - if (cacheattrs != NULL) { + if (mmu_idx == ARMMMUIdx_Stage2) { + memattr = convert_stage2_attrs(env, extract32(attrs, 0, 4)); + } else { + /* Index into MAIR registers for cache attributes */ + uint64_t mair = env->cp15.mair_el[el]; + memattr = extract64(mair, extract32(attrs, 0, 3) * 8, 8); + } + + /* When MTE is enabled, remember Tagged Memory in IOTLB. */ + if (aarch64 && cpu_isar_feature(aa64_mte, cpu)) { if (mmu_idx == ARMMMUIdx_Stage2) { - cacheattrs->attrs = convert_stage2_attrs(env, - extract32(attrs, 0, 4)); + /* + * Require Normal, I+O Shareable, WB, NT, RA, WA (0xff). + * If not, squash stage1 tagged normal setting. + */ + if (memattr != 0xff) { + txattrs->target_tlb_bit1 = false; + } } else { - /* Index into MAIR registers for cache attributes */ - uint8_t attrindx = extract32(attrs, 0, 3); - uint64_t mair = env->cp15.mair_el[regime_el(env, mmu_idx)]; - assert(attrindx <= 7); - cacheattrs->attrs = extract64(mair, attrindx * 8, 8); + /* Tagged Normal Memory (0xf0). */ + txattrs->target_tlb_bit1 = (memattr == 0xf0); } + } + + if (cacheattrs != NULL) { + cacheattrs->attrs = memattr; cacheattrs->shareability = extract32(attrs, 6, 2); } @@ -12065,6 +12081,17 @@ bool get_phys_addr(CPUARMState *env, target_ulong address, *phys_ptr = address; *prot = PAGE_READ | PAGE_WRITE | PAGE_EXEC; *page_size = TARGET_PAGE_SIZE; + + /* Stage1 translations are Tagged or Untagged based on HCR_DCT. */ + if (cpu_isar_feature(aa64_mte, env_archcpu(env)) + && (mmu_idx == ARMMMUIdx_Stage1_E0 || + mmu_idx == ARMMMUIdx_Stage1_E1 || + mmu_idx == ARMMMUIdx_Stage1_E1_PAN)) { + uint64_t hcr = arm_hcr_el2_eff(env); + if ((hcr & (HCR_DC | HCR_DCT)) == (HCR_DC | HCR_DCT)) { + attrs->target_tlb_bit1 = true; + } + } return 0; } From patchwork Wed Jun 3 01:13:16 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 187437 Delivered-To: patch@linaro.org Received: by 2002:a92:cf06:0:0:0:0:0 with SMTP id c6csp80249ilo; Tue, 2 Jun 2020 18:31:41 -0700 (PDT) X-Google-Smtp-Source: ABdhPJxHlJB+U8343BRQm4xoCQFaKVZ1Lv48fl5L3dxKxr7q5JYy795nvgpJN5usMOQlUtEpry0C X-Received: by 2002:a25:6d0a:: with SMTP id i10mr49137145ybc.193.1591147901134; Tue, 02 Jun 2020 18:31:41 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1591147901; cv=none; d=google.com; s=arc-20160816; b=YPe+cJXUCUpV6uUmvy/lozgs/lw5qng7pVGLDA6iMbSfyDhleif/xzk3SRSFZn7K6o S7YkdLnvGc/vRwrmU9y/9EHPATdIsrosslR0fDhcSebSvVj8OAHkpX4x5YsM1hzUZpKC 8dC8fygpl5ZDVRshey4GCaT5rwvOSChEqYZL+MpGDDpXDZ9IVfDiSEs+NtPgtJLVwZoH hlAAiLf4NlC9iDZ7OQm4X82sgG2KLszK7096Gx6ZblrGILpcIobixzXeTmOZ1cP80bF3 m/GkrV81YDal1aJ5+OlfJ8waTjzQKYh31xfB9Fh3fzobkQl+BNrHxlyW+Ipv+pr1BWfL ie7g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=pJfY0SD5Bvg4zcu/wLj7Ethd3vdK/dfygcX+yAzAWwA=; b=K9wpFWVwnPJV1nstqE74ZO001VELtdUtn1FQNF1sK/wOwLMe5g6XpPsXdLlnSYhHDQ pkyxJ/mlC9YlH+WfaoufvPsxoSTEvlC7chdRGE8T4fnQ8F/buu7NQLJCSouMVfp9Vwsu NddF+aDR5EI9nsxYkSX2wPOeeLsWxVxEcEQ98JFS3x85ixwvmqeCN9kcLEUAeqrEF9O2 AGnbCNh8Y26R0Y7b83QGTaeVBJUhW73EEPE6zK2ahcazpfXrC5OnFZ93+Z2TTIP1RRz7 i+vtr8LhPF+DBSkTdKps7sBnrnhAkwGPeaiOlSBLR9ArbUPRscuqrU4IAIePK4+1llI/ 9AOA== ARC-Authentication-Results: i=1; mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=mmp5lRt3; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id w6si445638ybt.180.2020.06.02.18.31.41 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 02 Jun 2020 18:31:41 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=mmp5lRt3; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:47856 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jgIFo-0005Ww-Ez for patch@linaro.org; Tue, 02 Jun 2020 21:31:40 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:49102) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jgHyx-0005Nx-G4 for qemu-devel@nongnu.org; Tue, 02 Jun 2020 21:14:15 -0400 Received: from mail-pg1-x544.google.com ([2607:f8b0:4864:20::544]:44494) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1jgHyw-0003iC-Bl for qemu-devel@nongnu.org; Tue, 02 Jun 2020 21:14:15 -0400 Received: by mail-pg1-x544.google.com with SMTP id p30so526560pgl.11 for ; Tue, 02 Jun 2020 18:14:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=pJfY0SD5Bvg4zcu/wLj7Ethd3vdK/dfygcX+yAzAWwA=; b=mmp5lRt3osAcx6TYFx/zaDZMVAWtr4g5vJ21BqwyqI/6hEjZR+CJH7PLrbSRRT7V4Q cQoteJ4MTtKe0TbI+Cd9C7qeyIZuwJpYT6cy0U4ZfVZjVLOKU4Dh85XGtR/VvXFJrrwg ityZISOOlnnjYVs1brt2+LTnvBTPA4s5rJtyrvGBfzrj73Vib/sV0yaq5MMJmBUfxFwS jqRxQA4QghZGOe6e8Mc8dsvvMEqxvos/CTcqQHEvrwP4AAHSpMtK08xube7R9Tv1zEcL zMmKq7k2vbjgQO/DdjsEQYPmJcPBMDHRYv/8Fc6+S0dc8p1T+2Wgr9xH58TGtsqEOGXh 3P8g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=pJfY0SD5Bvg4zcu/wLj7Ethd3vdK/dfygcX+yAzAWwA=; b=mslfA4v21IWic9a6YWDzSJF/7PZOQvpezbitTwGbRQIBqaiXxTh+ulac+DSNAQGyCk 8BfBEqBRiQKtVkRfcGkuYvC8Z9Q5vHHnVraQiAyOQAPhirrWG31pJZxz3W4PQd2GDCe9 QiGR8+G53Cl+cGy93Sbnfv4fAojV6dFsgl/kburBaAeYZNaWnYpv0qk/oMJZSkSfB/EM lX92jY9hJ/qmm2gwQlaF3W5oso2UGcRMBe/ia/oO7krNZyjqDrOnrAk1b2kx7YtFlMML LiPIpHoJ3I3kvL3oWomDygYn2aRJbJobsC1arlRs52fDNv3877K6GC3aNGC8ZYLgYi3L jhVw== X-Gm-Message-State: AOAM533Pl0XfSjXg9sT/aCUPl01Nz9h1nFvd8xh6zSJA2OgGVjGdywpN mksHiF8kHSLcoPzTnHB7e8B8BHK6U1I= X-Received: by 2002:a17:90a:950e:: with SMTP id t14mr2112024pjo.99.1591146852605; Tue, 02 Jun 2020 18:14:12 -0700 (PDT) Received: from localhost.localdomain (174-21-143-238.tukw.qwest.net. [174.21.143.238]) by smtp.gmail.com with ESMTPSA id 3sm290067pfe.85.2020.06.02.18.14.11 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 02 Jun 2020 18:14:12 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v7 41/42] target/arm: Create tagged ram when MTE is enabled Date: Tue, 2 Jun 2020 18:13:16 -0700 Message-Id: <20200603011317.473934-42-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200603011317.473934-1-richard.henderson@linaro.org> References: <20200603011317.473934-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::544; envelope-from=richard.henderson@linaro.org; helo=mail-pg1-x544.google.com X-detected-operating-system: by eggs.gnu.org: No matching host in p0f cache. That's all we know. X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: peter.maydell@linaro.org, qemu-arm@nongnu.org, steplong@quicinc.com Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" Signed-off-by: Richard Henderson --- v5: Assign cs->num_ases to the final value first. Downgrade to ID_AA64PFR1.MTE=1 if tag memory is not available. v6: Add secure tag memory for EL3. --- target/arm/cpu.h | 6 ++++++ hw/arm/virt.c | 52 ++++++++++++++++++++++++++++++++++++++++++++++++ target/arm/cpu.c | 51 ++++++++++++++++++++++++++++++++++++++++++++--- 3 files changed, 106 insertions(+), 3 deletions(-) -- 2.25.1 diff --git a/target/arm/cpu.h b/target/arm/cpu.h index 17594226eb..deb270f25b 100644 --- a/target/arm/cpu.h +++ b/target/arm/cpu.h @@ -792,6 +792,10 @@ struct ARMCPU { /* MemoryRegion to use for secure physical accesses */ MemoryRegion *secure_memory; + /* MemoryRegion to use for allocation tag accesses */ + MemoryRegion *tag_memory; + MemoryRegion *secure_tag_memory; + /* For v8M, pointer to the IDAU interface provided by board/SoC */ Object *idau; @@ -2985,6 +2989,8 @@ typedef enum ARMMMUIdxBit { typedef enum ARMASIdx { ARMASIdx_NS = 0, ARMASIdx_S = 1, + ARMASIdx_TagNS = 2, + ARMASIdx_TagS = 3, } ARMASIdx; /* Return the Exception Level targeted by debug exceptions. */ diff --git a/hw/arm/virt.c b/hw/arm/virt.c index 37462a6f78..c74cc3d78d 100644 --- a/hw/arm/virt.c +++ b/hw/arm/virt.c @@ -1412,6 +1412,16 @@ static void create_secure_ram(VirtMachineState *vms, g_free(nodename); } +static void create_tag_ram(MemoryRegion *tag_sysmem, + hwaddr base, hwaddr size, + const char *name) +{ + MemoryRegion *tagram = g_new(MemoryRegion, 1); + + memory_region_init_ram(tagram, NULL, name, size / 32, &error_fatal); + memory_region_add_subregion(tag_sysmem, base / 32, tagram); +} + static void *machvirt_dtb(const struct arm_boot_info *binfo, int *fdt_size) { const VirtMachineState *board = container_of(binfo, VirtMachineState, @@ -1665,6 +1675,8 @@ static void machvirt_init(MachineState *machine) const CPUArchIdList *possible_cpus; MemoryRegion *sysmem = get_system_memory(); MemoryRegion *secure_sysmem = NULL; + MemoryRegion *tag_sysmem = NULL; + MemoryRegion *secure_tag_sysmem = NULL; int n, virt_max_cpus; bool firmware_loaded; bool aarch64 = true; @@ -1819,6 +1831,35 @@ static void machvirt_init(MachineState *machine) "secure-memory", &error_abort); } + /* + * The cpu adds the property if and only if MemTag is supported. + * If it is, we must allocate the ram to back that up. + */ + if (object_property_find(cpuobj, "tag-memory", NULL)) { + if (!tag_sysmem) { + tag_sysmem = g_new(MemoryRegion, 1); + memory_region_init(tag_sysmem, OBJECT(machine), + "tag-memory", UINT64_MAX / 32); + + if (vms->secure) { + secure_tag_sysmem = g_new(MemoryRegion, 1); + memory_region_init(secure_tag_sysmem, OBJECT(machine), + "secure-tag-memory", UINT64_MAX / 32); + + /* As with ram, secure-tag takes precedence over tag. */ + memory_region_add_subregion_overlap(secure_tag_sysmem, 0, + tag_sysmem, -1); + } + } + + object_property_set_link(cpuobj, OBJECT(tag_sysmem), + "tag-memory", &error_abort); + if (vms->secure) { + object_property_set_link(cpuobj, OBJECT(secure_tag_sysmem), + "secure-tag-memory", &error_abort); + } + } + object_property_set_bool(cpuobj, true, "realized", &error_fatal); object_unref(cpuobj); } @@ -1861,6 +1902,17 @@ static void machvirt_init(MachineState *machine) create_uart(vms, VIRT_SECURE_UART, secure_sysmem, serial_hd(1)); } + if (tag_sysmem) { + create_tag_ram(tag_sysmem, vms->memmap[VIRT_MEM].base, + machine->ram_size, "mach-virt.tag"); + if (vms->secure) { + create_tag_ram(secure_tag_sysmem, + vms->memmap[VIRT_SECURE_MEM].base, + vms->memmap[VIRT_SECURE_MEM].size, + "mach-virt.secure-tag"); + } + } + vms->highmem_ecam &= vms->highmem && (!firmware_loaded || aarch64); create_rtc(vms); diff --git a/target/arm/cpu.c b/target/arm/cpu.c index 8e6d4371cb..37b7cc2c9c 100644 --- a/target/arm/cpu.c +++ b/target/arm/cpu.c @@ -1245,6 +1245,25 @@ void arm_cpu_post_init(Object *obj) if (arm_feature(&cpu->env, ARM_FEATURE_GENERIC_TIMER)) { qdev_property_add_static(DEVICE(cpu), &arm_cpu_gt_cntfrq_property); } + +#ifndef CONFIG_USER_ONLY + if (arm_feature(&cpu->env, ARM_FEATURE_AARCH64) && + cpu_isar_feature(aa64_mte, cpu)) { + object_property_add_link(obj, "tag-memory", + TYPE_MEMORY_REGION, + (Object **)&cpu->tag_memory, + qdev_prop_allow_set_link_before_realize, + OBJ_PROP_LINK_STRONG); + + if (arm_feature(&cpu->env, ARM_FEATURE_EL3)) { + object_property_add_link(obj, "secure-tag-memory", + TYPE_MEMORY_REGION, + (Object **)&cpu->secure_tag_memory, + qdev_prop_allow_set_link_before_realize, + OBJ_PROP_LINK_STRONG); + } + } +#endif } static void arm_cpu_finalizefn(Object *obj) @@ -1735,17 +1754,43 @@ static void arm_cpu_realizefn(DeviceState *dev, Error **errp) MachineState *ms = MACHINE(qdev_get_machine()); unsigned int smp_cpus = ms->smp.cpus; - if (cpu->has_el3 || arm_feature(env, ARM_FEATURE_M_SECURITY)) { + /* + * We must set cs->num_ases to the final value before + * the first call to cpu_address_space_init. + */ + if (cpu->tag_memory != NULL) { + cs->num_ases = 4; + } else if (cpu->has_el3 || arm_feature(env, ARM_FEATURE_M_SECURITY)) { cs->num_ases = 2; + } else { + cs->num_ases = 1; + } + if (cpu->has_el3 || arm_feature(env, ARM_FEATURE_M_SECURITY)) { if (!cpu->secure_memory) { cpu->secure_memory = cs->memory; } cpu_address_space_init(cs, ARMASIdx_S, "cpu-secure-memory", cpu->secure_memory); - } else { - cs->num_ases = 1; } + + if (cpu->tag_memory != NULL) { + cpu_address_space_init(cs, ARMASIdx_TagNS, "cpu-tag-memory", + cpu->tag_memory); + if (cpu->has_el3) { + cpu_address_space_init(cs, ARMASIdx_TagS, "cpu-tag-memory", + cpu->secure_tag_memory); + } + } else if (cpu_isar_feature(aa64_mte, cpu)) { + /* + * Since there is no tag memory, we can't meaningfully support MTE + * to its fullest. To avoid problems later, when we would come to + * use the tag memory, downgrade support to insns only. + */ + cpu->isar.id_aa64pfr1 = + FIELD_DP64(cpu->isar.id_aa64pfr1, ID_AA64PFR1, MTE, 1); + } + cpu_address_space_init(cs, ARMASIdx_NS, "cpu-memory", cs->memory); /* No core_count specified, default to smp_cpus. */ From patchwork Wed Jun 3 01:13:17 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 187439 Delivered-To: patch@linaro.org Received: by 2002:a92:cf06:0:0:0:0:0 with SMTP id c6csp80525ilo; Tue, 2 Jun 2020 18:32:10 -0700 (PDT) X-Google-Smtp-Source: ABdhPJxfwFCaBU7OLiNUb3w6RKE+IU6kiaN4mm8PokE0f0fGfJ0GJLxMvN4mFR1hJhIfI5BtxJYb X-Received: by 2002:a25:b882:: with SMTP id w2mr12429744ybj.160.1591147930471; Tue, 02 Jun 2020 18:32:10 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1591147930; cv=none; d=google.com; s=arc-20160816; b=X799gem95xTpV2mbqZslfE1VdVdpR5lltBMoH3+snGiCHvaGITyyZaZpqa5R7tSlZ6 VMHVhb9Fnem93ZUb6AMQZ41QxeZovBgLKyf4HezfktVS0UDvuuT5AUo88PY5Gqy6CN2U B1EsAfJNiKG96VG5UwH948QOQXvqw66eCa+dF2zITXm1AFW5jplWrsdPJYpsVaprqZSd CvzW+aI6Ox42Tml0iw9t2epuxE2VFFJ1MO56W1Fp918zgexG5BsETGcBfiTniuLtk5Pa 2Qlgs0DqG/5KR1uZ17KuhLq2cyEyJF1OXj4WYPJF/c1DJjDHPtq/cD0HrtGqeY2OvWrB Zfiw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=J2ezAdBRDnrkWMhAyjiFD4czb9AMeAMbprziLRhYAWw=; b=jbc0oNOOipMvbCXH9FW1k2VlZslUSp4mxdPgNhVS/H1S5OXT4iv90r9yd61moyumrU +Z1Z94NT1O38OJ7ikCz+XW7hl5uckq7dsuwp1iW/2pchErsGSg+zQNcnKldQ8OJLABMM oyazP3mmc6tWOgMa+LtN1N2ii36ue6NRBFZbQgnfoPGZqrV17ZiiUiOe0F22VclzTq6O dqU1sdDGH4VHdQ89u3NoRxacvNm/ysDgmIayRvRjFQKkkBXJsUq46hX9B3PQki0bmKL/ 72xIMCfYxn806HLbibm/vBh67AT8Who+uMmtaPg0PNaghZVPlZkLCEMSt0AGm8+DXHHE zk6Q== ARC-Authentication-Results: i=1; mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=wW+mQZsk; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id c195si334495ybf.31.2020.06.02.18.32.10 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 02 Jun 2020 18:32:10 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=wW+mQZsk; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:50394 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jgIGH-0006Wg-UA for patch@linaro.org; Tue, 02 Jun 2020 21:32:09 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:49120) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jgHyy-0005R1-HE for qemu-devel@nongnu.org; Tue, 02 Jun 2020 21:14:16 -0400 Received: from mail-pg1-x541.google.com ([2607:f8b0:4864:20::541]:35463) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1jgHyx-0003iP-JV for qemu-devel@nongnu.org; Tue, 02 Jun 2020 21:14:16 -0400 Received: by mail-pg1-x541.google.com with SMTP id o6so580221pgh.2 for ; Tue, 02 Jun 2020 18:14:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=J2ezAdBRDnrkWMhAyjiFD4czb9AMeAMbprziLRhYAWw=; b=wW+mQZskGhF9jBDWsZKyoSp0hDN7MpgKf48LqKuR1gDhOU09lBFu1ycQoH2TkBuRRa ZzI7atoNYckPFuB9oTAM3CkPmBKnylzlRjgO7R/+3M+7XDVTcCpXd2q1U4vG3Eit4jfy kcvSoenUYqi/1uMTZSL7ZdfpAxdfbWoFmT1peQxieCUYwUo7lpA1DdNBVxDbTtJcwAJo 7gZzeOq7omQ9THXVifbQcMCnV8HKCyKcU2HiHdXcyYmBpxwkoLQxGHxsWDMcmo9XqWUR uSNkj+RO2MWiU4ugg4Xb/5sWcZflJ5+IXp9qKOw8MV3uOAzcZnX0SauRhzh9d/U8B7z5 puUA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=J2ezAdBRDnrkWMhAyjiFD4czb9AMeAMbprziLRhYAWw=; b=KEiIzjauZBpSfm7TZUdSKpv6/0uc3+fLKbvi9WKhIejfaK1g5NCYYx8i+70WPhUjCR XMjzm1FlNdKkmoPKKJcx39AqU1eXP8fTDAM1IfomKjO9fCzliAFW3MxxvBsnqaLYFig2 XUUtDSqyMViXB1ydyn48ql7JGCoxp4hYEKRH5xoE4Hfs6cSSr4C1Hk1djt1BtptJLF7q JhzgIq8Edl9U/ueERR41IYPwTjU+B5myDVzBpNKwzOSQ2ElWSijJt8gXnV1PQliH8Vli LlBAXu1YbkFRkLlu/P3ZL3SyAL/NLa0bVzlwnyJ7/QvQDTAWt7BnFuqYnSIaUeB65BpO t9rg== X-Gm-Message-State: AOAM533lb3rIKOO9rAQgSJGdLMbknlXIu0w6JUrp+YYU3WCCmwdwzl8/ dccJNTpC/KfrN+Oo6flKNWH4LRLbkw4= X-Received: by 2002:a17:90a:bf18:: with SMTP id c24mr2327985pjs.171.1591146853827; Tue, 02 Jun 2020 18:14:13 -0700 (PDT) Received: from localhost.localdomain (174-21-143-238.tukw.qwest.net. [174.21.143.238]) by smtp.gmail.com with ESMTPSA id 3sm290067pfe.85.2020.06.02.18.14.12 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 02 Jun 2020 18:14:13 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v7 42/42] target/arm: Add allocation tag storage for system mode Date: Tue, 2 Jun 2020 18:13:17 -0700 Message-Id: <20200603011317.473934-43-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200603011317.473934-1-richard.henderson@linaro.org> References: <20200603011317.473934-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::541; envelope-from=richard.henderson@linaro.org; helo=mail-pg1-x541.google.com X-detected-operating-system: by eggs.gnu.org: No matching host in p0f cache. That's all we know. X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: peter.maydell@linaro.org, qemu-arm@nongnu.org, steplong@quicinc.com Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" Look up the physical address for the given virtual address, convert that to a tag physical address, and finally return the host address that backs it. Signed-off-by: Richard Henderson --- target/arm/mte_helper.c | 128 ++++++++++++++++++++++++++++++++++++++++ 1 file changed, 128 insertions(+) -- 2.25.1 diff --git a/target/arm/mte_helper.c b/target/arm/mte_helper.c index 175da11e01..b15fa665df 100644 --- a/target/arm/mte_helper.c +++ b/target/arm/mte_helper.c @@ -21,6 +21,7 @@ #include "cpu.h" #include "internals.h" #include "exec/exec-all.h" +#include "exec/ram_addr.h" #include "exec/cpu_ldst.h" #include "exec/helper-proto.h" @@ -74,8 +75,135 @@ static uint8_t *allocation_tag_mem(CPUARMState *env, int ptr_mmu_idx, int ptr_size, MMUAccessType tag_access, int tag_size, uintptr_t ra) { +#ifdef CONFIG_USER_ONLY /* Tag storage not implemented. */ return NULL; +#else + uintptr_t index; + CPUIOTLBEntry *iotlbentry; + int in_page, flags; + ram_addr_t ptr_ra; + hwaddr ptr_paddr, tag_paddr, xlat; + MemoryRegion *mr; + ARMASIdx tag_asi; + AddressSpace *tag_as; + void *host; + + /* + * The caller must split calls to this function such that it will + * not access *tag* memory beyond the end of the page. + */ + in_page = -(ptr | -(TARGET_PAGE_SIZE >> (LOG2_TAG_GRANULE + 1))); + g_assert(tag_size <= in_page); + + /* + * Probe the first byte of the virtual address. This raises an + * exception for inaccessible pages, and resolves the virtual address + * into the softmmu tlb. + */ + flags = probe_access_flags(env, ptr, ptr_access, ptr_mmu_idx, + false, &host, ra); + + /* + * Find the iotlbentry for ptr. This *must* be present in the TLB + * because we just found the mapping. + * TODO: Perhaps there should be a cputlb helper that returns a + * matching tlb entry + iotlb entry. + */ + index = tlb_index(env, ptr_mmu_idx, ptr); +# ifdef CONFIG_DEBUG_TCG + { + CPUTLBEntry *entry = tlb_entry(env, ptr_mmu_idx, ptr); + target_ulong comparator = (ptr_access == MMU_DATA_LOAD + ? entry->addr_read + : tlb_addr_write(entry)); + g_assert(tlb_hit(comparator, ptr)); + } +# endif + iotlbentry = &env_tlb(env)->d[ptr_mmu_idx].iotlb[index]; + + /* If the virtual page MemAttr != Tagged, access unchecked. */ + if (!iotlbentry->attrs.target_tlb_bit1) { + return NULL; + } + + /* If not normal memory, tag storage is not implemented, access unchecked. */ + if (unlikely(flags & TLB_MMIO)) { + qemu_log_mask(LOG_GUEST_ERROR, + "Page @ 0x%" PRIx64 " indicates Tagged Normal memory " + "but is Device memory\n", ptr); + return NULL; + } + + /* + * The Normal memory access can extend to the next page. E.g. a single + * 8-byte access to the last byte of a page will check only the last + * tag on the first page. + * Any page access exception has priority over tag check exception. + */ + in_page = -(ptr | TARGET_PAGE_MASK); + if (unlikely(ptr_size > in_page)) { + void *ignore; + flags |= probe_access_flags(env, ptr + in_page, ptr_access, + ptr_mmu_idx, false, &ignore, ra); + } + + /* Any debug exception has priority over a tag check exception. */ + if (unlikely(flags & TLB_WATCHPOINT)) { + int wp = ptr_access == MMU_DATA_LOAD ? BP_MEM_READ : BP_MEM_WRITE; + cpu_check_watchpoint(env_cpu(env), ptr, ptr_size, + iotlbentry->attrs, wp, ra); + } + + /* + * Find the physical address within the normal mem space. + * The memory region lookup must succeed because TLB_MMIO was + * not set in the cputlb lookup above. + */ + mr = memory_region_from_host(host, &ptr_ra); + tcg_debug_assert(mr != NULL); + tcg_debug_assert(memory_region_is_ram(mr)); + ptr_paddr = ptr_ra; + do { + ptr_paddr += mr->addr; + mr = mr->container; + } while (mr); + + /* Convert to the physical address in tag space. */ + tag_paddr = ptr_paddr >> (LOG2_TAG_GRANULE + 1); + + /* Look up the address in tag space. */ + tag_asi = iotlbentry->attrs.secure ? ARMASIdx_TagS : ARMASIdx_TagNS; + tag_as = cpu_get_address_space(env_cpu(env), tag_asi); + mr = address_space_translate(tag_as, tag_paddr, &xlat, NULL, + tag_access == MMU_DATA_STORE, + iotlbentry->attrs); + + /* + * Note that @mr will never be NULL. If there is nothing in the address + * space at @tag_paddr, the translation will return the unallocated memory + * region. For our purposes, the result must be ram. + */ + if (unlikely(!memory_region_is_ram(mr))) { + /* ??? Failure is a board configuration error. */ + qemu_log_mask(LOG_UNIMP, + "Tag Memory @ 0x%" HWADDR_PRIx " not found for " + "Normal Memory @ 0x%" HWADDR_PRIx "\n", + tag_paddr, ptr_paddr); + return NULL; + } + + /* + * Ensure the tag memory is dirty on write, for migration. + * Tag memory can never contain code or display memory (vga). + */ + if (tag_access == MMU_DATA_STORE) { + ram_addr_t tag_ra = memory_region_get_ram_addr(mr) + xlat; + cpu_physical_memory_set_dirty_flag(tag_ra, DIRTY_MEMORY_MIGRATION); + } + + return memory_region_get_ram_ptr(mr) + xlat; +#endif } uint64_t HELPER(irg)(CPUARMState *env, uint64_t rn, uint64_t rm)